WO2017061239A1

WO2017061239A1 - Surveillance system

Info

Publication number: WO2017061239A1
Application number: PCT/JP2016/076955
Authority: WO
Inventors: 平井　誠一
Original assignee: 株式会社日立国際電気
Priority date: 2015-10-09
Filing date: 2016-09-13
Publication date: 2017-04-13
Also published as: JPWO2017061239A1; JP6632632B2

Abstract

The objective of the present invention is to reduce network flow rates. The surveillance system according to the present invention is provided with an image capturing device, a user image capturing device and a terminal device, and is characterized in that: the image capturing device includes a means for changing the image quality and the frame rate of a delivered image; the user image capturing device captures images of a surveillant of the terminal device; and the terminal device includes a means for acquiring, from the images captured by the user image capturing device, a gazing frequency of the surveillant with respect to the images, a means for setting the image quality or the frame rate of the delivered image on the basis of the gazing frequency, and a means for outputting a delivery instruction to the image capturing device on the basis of the determined result.

Description

Monitoring system

The present invention relates to a monitoring system.

Conventionally, surveillance systems are installed in facilities visited by an unspecified number of people such as hotels, buildings, convenience stores, financial institutions, dams and roads for the purpose of crime prevention and accident prevention. In such a monitoring system, a person to be monitored is photographed by an imaging device such as a camera, and the photographed image is transmitted to a monitoring center such as a management office or a security room, and the resident supervisor monitors it. Then, it calls attention or records video according to the purpose and necessity.
In recent years, with the worsening of security and increasing social unrest, the price of surveillance cameras has been reduced, such monitoring systems have been developed not only for the facilities described above, but also for shopping streets, downtowns, and residential roads in residential areas. Increasing cases are being installed.

As a prior art document, for example, Patent Document 1 discloses an invention that can seamlessly use various communication forms such as voice communication, video communication, and the middle thereof with low burden.
As another prior art document, for example, Patent Document 2 discloses an invention in which a best shot is reliably recorded without missing a facial expression of a person.

Patent Registration No. 4475579 JP 2010-177859 A

The expansion of the installation location of the monitoring system dramatically increases the number of imaging devices connected to the system, and increases the network flow rate, which increases the cost for constructing and maintaining the network.
An object of the present invention is to reduce network flow.

The monitoring system of the present invention is a monitoring system including an imaging device, a user imaging device, and a terminal device, the imaging device has means for changing the image quality and frame rate of the distribution image, and the user imaging device is a terminal device. The monitoring device is photographed, and the terminal device acquires a gaze frequency for the monitor image from the captured image of the user imaging device, a unit for determining the image quality or frame rate of the distribution image based on the gaze frequency, and the determination result And a means for issuing a delivery instruction to the imaging device based on the above.

The monitoring system of the present invention is a monitoring system including an imaging device, a user imaging device, and a terminal device. The imaging device has means for changing the image quality and frame rate of a distribution image, and the user imaging device is a terminal. The terminal image is taken by the terminal device, the terminal device displays the distribution image from the imaging device, the means for personal identification of the supervisor from the captured image of the user imaging device, and the image quality of the distribution image based on the personal identification result And a means for determining a frame rate and a means for issuing a distribution instruction to the imaging apparatus based on the determined result.

According to the present invention, the network flow rate can be reduced.

It is a figure which shows the structure of the monitoring system which concerns on one Example of this invention. It is a figure which shows the function structure of the monitoring system which concerns on one Example of this invention. It is a figure which shows the aspect of the delivery instruction | indication data in the delivery instruction | indication part of the monitoring system which concerns on one Example of this invention. It is a flowchart for demonstrating the flow of a process in the terminal device of the monitoring system which concerns on one Example of this invention. It is a figure which shows the structure of the monitoring system which concerns on another Example of this invention. It is a figure which shows the function structure of the monitoring system which concerns on another one Example of this invention. It is a figure which shows the aspect of the person data in the person recording part of the monitoring system which concerns on another one Example of this invention. It is a flowchart for demonstrating the flow of a process in the terminal device of the monitoring system which concerns on another one Example of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. First Embodiment A first embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is a diagram showing a configuration of a monitoring system according to an embodiment of the present invention.
In FIG. 1, the monitoring system includes a network 100, monitoring imaging devices 101 to 105, a terminal device 106, and a user imaging device 107.

The network 100 includes a dedicated line such as a USB (Universal Serial Bus) that connects the monitoring imaging devices 101 to 105, the terminal device 106, and the user imaging device 107 to each other for data communication, an intranet, the Internet, and a wireless LAN (Local Area Network). ).
The surveillance imaging devices 101 to 105 include a zoom lens capable of perspective control and focus control, an imaging device such as a CCD or CMOS, an A / D circuit for digitally converting an analog signal, a temporary storage memory such as a RAM, a data transmission bus, It is a device such as a monitoring camera equipped with a timing circuit, an external input / output interface, a power supply circuit, a pan / tilt head, illumination such as a visible light and a near infrared LED (Light Emitting Diode).

The monitoring imaging devices 101 to 105 convert the light passing through the lens into an electrical signal by the imaging device, digitally convert the electrical signal by the A / D circuit, and store (store) it as image data in the temporary storage memory. The stored image data is output from the external input / output interface to the network in response to an external video request input to the external input / output interface or an instruction from the timing circuit. Similarly, it is possible to change the image quality of the image to be distributed, the distribution frame rate, and the like in accordance with an external control command input to the external input / output interface.

The terminal device 106 includes an arithmetic circuit such as a CPU (Central Processing Unit), a temporary storage memory such as a RAM (Random Access Memory), a recording medium such as an HDD (Hard Disk Drive), a data transmission bus, an external input / output interface, a power supply A device such as a computer provided with a circuit, a screen such as a liquid crystal display, and user input devices such as a keyboard and a mouse.
The terminal device 106 stores the image data from the monitoring imaging devices 101 to 105 input from the network 100 to the external input / output interface in the temporary storage memory. The stored image data is converted into a form suitable for display using an arithmetic circuit and displayed on the screen. The recording medium stores a set of software including the method of the present invention, an OS (operation system), and the like. In addition, a user operation on the terminal device 106 is performed on a user input device.

The user imaging device 107 includes a zoom lens capable of perspective control and focus control, an imaging device such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor), an A / D circuit that converts an analog signal into digital, a RAM, and the like. Devices such as a USB camera equipped with a temporary storage memory, a data transmission bus, a timing circuit, an external input / output interface, a power supply circuit, and the like.

The user imaging device 107 converts the light that has passed through the lens into an electrical signal by the imaging device, digitally converts the electrical signal by the A / D circuit, and stores it as image data in the temporary storage memory. The stored image data is output from the external input / output interface to the network in response to an external video request input to the external input / output interface or an instruction from the timing circuit. Similarly, it is possible to change the image quality of the image to be distributed, the distribution frame rate, and the like in accordance with an external control command input to the external input / output interface.
The user imaging device 107 is arranged at a position where the user's (monitor) face can be photographed from the front, for example, near the screen of the terminal device 106.

Next, the functional configuration of the monitoring system according to an embodiment of the present invention will be described with reference to FIG. FIG. 2 is a diagram showing a functional configuration of the monitoring system according to one embodiment of the present invention. The same reference numerals as those in FIG. 1 denote the same devices.
The terminal device 106 includes a distribution instruction unit 200, a monitoring image reception unit 201, a screen display unit 202, a user image acquisition unit 203, a face detection unit 204, a gaze determination unit 205, a determination result holding unit 206, and a control unit 207. Yes.

The distribution instruction unit 200 is a functional unit that instructs distribution of monitoring images to the monitoring imaging devices 101 to 105. The distribution instruction includes an instruction on a distribution frame rate and image quality. The distribution instruction unit 200 holds distribution instruction data. The mode of the distribution instruction data will be described later.
The monitoring image receiving unit 201 is a functional unit that acquires a monitoring image from the monitoring imaging device. In this embodiment, image data input from the monitoring imaging devices 101 to 105 is received.

The screen display unit 202 is a functional unit that displays a monitor image on the screen. In the present embodiment, the image data received by the image receiving unit 201 is converted into a form suitable for display and displayed.
The user image acquisition unit 203 is a functional unit that acquires a user (monitorer) image from the user imaging device. In this embodiment, image data is acquired from the user imaging device 107.
The face detection unit 204 is a functional unit that detects the face of the supervisor from the user image. In the present embodiment, face detection using image recognition technology is performed on the image data acquired by the user image acquisition unit 203, the presence of a monitor is determined, and if the monitor exists, the region coordinate output is output. Do.

The gaze determination unit 205 is a functional unit that determines the presence or absence of a monitoring image of the monitoring person and the gaze target image. In the present embodiment, when the face detection unit 204 determines that there is a monitor, the direction of the line of sight is detected, and when the point of the line of sight is a monitoring image displayed by the screen display unit 202, the user is watched. It is determined that At that time, the monitoring image that is the gaze target is also determined.
The determination result holding unit 206 is a functional unit that holds the result determined by the gaze determination unit 205 for a predetermined time. In this embodiment, the result determined by the gaze determination unit 205 is stored in the temporary storage memory together with the determination time. The holding time is given as a preset, and the result exceeding the holding time is discarded. Any manner of holding may be used.
The control unit 207 is a functional unit that controls each functional unit of the terminal device 106.

Next, the mode of distribution instruction data in the distribution instruction unit will be described with reference to FIG.
FIG. 3 is a diagram showing an aspect of distribution instruction data in the distribution instruction unit of the monitoring system according to one embodiment of the present invention.
The distribution instruction data 300 stores one type of distribution instruction as one record. In this embodiment, seven records 301 to 307, that is, seven types of distribution instructions can be stored. This data is set in advance.
The record includes a frequency cell 310, a distribution frame rate cell 311, and an image quality cell 312.
The frequency cell 310 is an area for storing a gaze frequency range. The gaze frequency is the number of gazes in a certain time, for example, 1 minute. In this example, specific numerical values are not shown, but it is assumed that the range stored in the record 301 is the most frequent and the value stored in the record 307 is the least frequent.

The distribution frame rate cell 311 is an area for storing a distribution frame rate. The distribution frame rate is the number of images output from the imaging device in a certain time, for example, 1 second. In this embodiment, specific numerical values are not shown, but it is assumed that the value stored in the record 301 is the highest frame rate and the value stored in the record 307 is the lowest frame rate.
The image quality cell 312 is an area for storing image quality. The image quality is constituted by, for example, image resolution, compression rate, or a combination thereof. In this embodiment, specific numerical values are not shown, but it is assumed that the value stored in the record 301 has the highest image quality and the value stored in the record 307 has the lowest image quality.

Next, the flow of processing in the terminal device 106 will be described with reference to FIG.
FIG. 4 is a flowchart for explaining the flow of processing in the terminal device of the monitoring system according to one embodiment of the present invention.
The processing in the terminal device 106 is composed of FIG. 4 (A) and FIG. 4 (B).
The process in FIG. 4B is performed for each of the monitoring imaging devices 101 to 105. In the system configuration of FIG. 1, the terminal device 106 is configured such that one process in FIG. 4A and five processes in FIG.

First, the flow of processing in FIG. 4A will be described.
In step 400, the control unit 207 of the terminal device 106 stands by for a predetermined time. The length of the waiting time is set in advance. After a predetermined time has elapsed, the process proceeds to step 401.
In step 401, the user image acquisition unit 203 of the terminal device 106 acquires a user image from the user imaging device 107.

In step 402, the face detection unit 204 of the terminal device 106 performs face detection on the user image data acquired in step 401. Face detection is, for example, a method for obtaining a difference from a background image and determining the area based on the area or shape of the difference area, or arrangement of main components such as eyes, nose, and mouth prepared and learned in advance. This is performed by the presence / absence of a region having an image feature pattern of the face such as the difference between the forehead and the contrast of the eyes, a search method using an in-image search, and the like. Any method may be used in an embodiment of the present invention.
If it is determined in step 402 that the face is detected (YES), the face detection unit 204 proceeds to step 403. If no face is detected (NO), the face detection unit 204 returns to step 400.

In step 403, the gaze determination unit 205 of the terminal device 106 detects organ positions such as the corners of the eyes, the head of the eyes, the center of the black eyes, the nose and the mouth of the face area obtained in step 402.
In step 404, the gaze determination unit 205 of the terminal device 106 detects the gaze direction. The line-of-sight direction is detected by a method of estimating the face direction from the positional relationship of the organs obtained in step 403 and determining from the positional relationship of the face direction and the black eye. Any method may be used in the present invention.

In step 405, the gaze determination unit 205 of the terminal device 106 determines the presence or absence of the monitoring image gaze. When the line of sight obtained in step 404 is directed to the monitoring image, it is determined that the monitoring image is being watched. If it is determined that there is a gaze (YES), the process proceeds to step 406. If it is determined that there is no gaze (NO), the process returns to step 400.
In step 406, the determination result holding unit 206 of the terminal device 106 stores (stores) the determination result obtained in step 405 and the determination time.

In step 407, the determination result holding unit 206 of the terminal device 106 calculates the gaze frequency in a predetermined time. The frequency is calculated from the number of determination results with gaze in a certain past time. After the process, the process returns to the process of step 400.

Next, the flow of processing in FIG. 4B will be described.
In step 410, the distribution instruction unit 200 of the terminal device 106 transmits a request for image distribution to the monitoring imaging device. At this time, for example, the requested distribution frame rate and image quality are taken out of the distribution instruction data 300 as default values stored in the record 304, for example. The distribution frame rate and image quality instructed in this step are stored (stored) as current distribution conditions.

In step 411, the monitoring image receiving unit 201 of the terminal device 106 waits for image reception, and when an incoming image from the monitoring imaging device 101 to 105 is detected, the process proceeds to step 412.
In step 412, the monitoring image receiving unit 201 of the terminal device 106 receives a monitoring image from the monitoring imaging device.

In step 413, the screen display unit 202 of the terminal device 106 performs decompression of the compressed image and conversion of the image format on the received monitoring image data as necessary, and displays them on the screen. At that time, the current delivery condition stored in step 410 is taken out and displayed in a large area if the condition is high quality, a small area if the condition is low, or a screen if the frame rate is high. Display control is also performed in accordance with distribution conditions such as displaying at the top of the screen, and displaying at the bottom of the screen when the frame rate is low. This display control is performed in a coordinated manner so that the display positions do not overlap among the five processes being executed in parallel.

In step 414, the distribution instruction unit 200 of the terminal device 106 extracts the gaze frequency calculated and stored in step 407 and refers to the distribution instruction data 300. For the records 301 to 307, the value stored in the frequency cell 310 and the gaze frequency extracted above are compared in order to find the corresponding record. Then, the distribution frame rate and image quality stored in the corresponding record are extracted. The extracted delivery frame rate and image quality are stored as new delivery conditions.

In step 415, the distribution instruction unit 200 of the terminal device 106 compares the current distribution condition with the new distribution condition. If they do not match (YES), the process proceeds to step 416. Otherwise (NO), the process returns to step 411.
In step 416, the distribution instruction unit 200 of the terminal apparatus 106 transmits an image distribution change request to the monitoring imaging apparatus. At that time, the requested distribution frame rate and image quality are the values extracted in step 414. The distribution frame rate and image quality instructed in this step are stored as current distribution conditions. After the process, the process returns to step 411.

As described so far, the present invention provides a method for obtaining a gaze frequency for a monitoring image in real time and performing distribution while changing a distribution condition according to the gaze frequency. As the gaze frequency is higher, the monitor image is displayed at a higher frame rate and higher image quality. Conversely, as the gaze frequency is lower, monitor image display is performed at a lower frame rate and lower image quality. Generally, the higher the frame rate and the higher image quality, the higher the network flow rate, and the lower the frame rate and the lower image quality, the lower the network flow rate. Therefore, the present invention makes it possible to optimize the network flow rate of the entire system.
In the description so far, for simplification of description, five monitoring imaging devices and one terminal device are shown in the configuration, but these are configurations other than the above for the network. Also good.

Second Embodiment Next, a second embodiment of the present invention will be described with reference to FIGS.
FIG. 5 is a diagram showing the configuration of a monitoring system according to another embodiment of the present invention.
5, devices having the same reference numerals as those in FIG. 1 indicate the same devices.
The monitoring system includes a network 100, monitoring imaging devices 101 to 105, a terminal device 506, and a user imaging device 107.

The terminal device 506 is an arithmetic circuit such as a CPU, a temporary storage memory such as a RAM, a recording medium such as an HDD, a data transmission bus, an external input / output interface, a power circuit, a screen such as a liquid crystal display, and a user input such as a keyboard and a mouse. It is a device such as a computer provided with a device.

The terminal device 506 stores (stores) the image data from the monitoring imaging devices 101 to 105 input from the network 100 to the external input / output interface in the temporary storage memory. The stored image data is converted into a form suitable for display using an arithmetic circuit and displayed on the screen. The recording medium stores a set of software including the method of the present invention, an OS (operation system), a set of images (person feature amounts) of a user (monitoring person), and the like. In addition, a user operation on the terminal device is performed on a user input device.
The user imaging device 107 is arranged at a position where the user (monitor) 's face can be photographed from the front, for example, near the screen of the terminal device 506.

Next, a functional configuration of a monitoring system according to another embodiment of the present invention will be described with reference to FIG.
FIG. 6 is a diagram showing a functional configuration of a monitoring system according to another embodiment of the present invention.
In FIG. 6, the same reference numerals as those in FIGS. 1, 2, and 5 denote the same devices.
The terminal device 506 includes a distribution instruction unit 200, a monitoring image reception unit 201, a screen display unit 202, a user image acquisition unit 203, a face detection unit 204, a face feature amount calculation unit 608, a person identification unit 609, a person recording unit 610, a control Each functional unit of the unit 611 is configured.

The face feature amount calculation unit 608 is a functional unit that calculates a feature amount using an image recognition technique for the face area in the image detected by the face detection unit 204.
The person identification unit 609 determines the person identification for the supervisor detected by the face detection unit 204 by comparing the feature amount obtained by the face feature amount calculation unit 608 with a database stored in the person recording unit 610. It is.

The person recording unit 610 is a functional unit that records the person feature amount of the supervisor, in this embodiment, the face feature amount together with the person ID (first and last name, identification information given to the person) and the like as person data. . The feature amount used here is an image feature amount for identifying an individual with respect to a person shown in the image, and may be, for example, the positional relationship of organs such as the eyes and nose and the respective contour information. In this example, any feature quantity may be used.
The person recording unit 610 records the delivery conditions for the monitoring imaging devices 101 to 105 for each person. Here, the distribution conditions are composed of a distribution frame rate, image quality, and the like. The person data is given in advance, and the way of giving it may be any. The aspect of the person data will be described later.
The control unit 611 is a functional unit that controls each functional unit of the terminal device 506.

Next, an aspect of person data in the person recording unit 610 will be described with reference to FIG.
FIG. 7 is a diagram showing an aspect of person data in the person recording unit of the monitoring system according to another embodiment of the present invention.
The person data 700 records information of one person as one record. The present embodiment shows information of three monitor persons, that is, a state where three records are recorded, and is composed of records 701 to 703. This data is set in advance.

The record is composed of a record number cell 710, a person ID cell 711, a feature amount cell 712, distribution frame rate cells 713 to 717, and image quality cells 718 to 722.
A record number cell 710 is an area for storing a record number. The record number is a number used for managing records, and is, for example, a continuous integer value uniquely assigned to each record.

The person ID cell 711 is an area for storing the person ID of the supervisor. In this embodiment, an example in which a character string as a person name is stored is shown. However, an integer value or the like may be used as long as it is identification information given to a person.
The feature quantity cell 712 is an area for storing the person feature quantity of the supervisor. In this embodiment, an example in which a decimal value as a one-dimensional value is stored is shown for ease of explanation, but a multidimensional value may be used.

Delivery frame rate cells 713 to 717 are areas for storing delivery frame rates. The distribution frame rate is the number of images output from the imaging device in a predetermined time, for example, 1 second. The distribution frame rate cell 713 is the distribution frame rate for the monitoring imaging apparatus 101, the distribution frame rate cell 714 is the distribution frame rate for the monitoring imaging apparatus 102, the distribution frame rate cell 715 is the distribution frame rate for the monitoring imaging apparatus 103, and the distribution frame. The rate cell 716 stores the distribution frame rate for the monitoring imaging device 104, and the distribution frame rate cell 717 stores the distribution frame rate for the monitoring imaging device 105, respectively. The value itself stored in the cell may be any value, and in this embodiment, the symbol is shown instead of a specific numerical value.

Image quality cells 718 to 722 are areas for storing image quality. The image quality is constituted by, for example, image resolution, compression rate, or a combination thereof. The value itself stored in this cell may be any value, and in this example, a symbol is shown instead of a specific numerical value.

Next, the flow of processing in the terminal device 506 will be described with reference to FIG.
FIG. 8 is a flowchart for explaining the flow of processing in the terminal device of the monitoring system according to another embodiment of the present invention.
The processing in the terminal device 506 is composed of FIG. 8A and FIG. Since the processing in FIG. 8B is performed for each monitoring imaging device, in the system configuration in FIG. 5 showing five monitoring imaging devices, inside the terminal device 506, FIG. There is one process, and a total of six processes in FIG. 5B are executed in parallel.

First, the flow of processing in FIG. 8A will be described.
In step 800, the control unit 611 of the terminal device 506 waits for a predetermined time. The length of the waiting time is set in advance. After a predetermined time elapses, the process proceeds to step 801.
In step 801, the user image acquisition unit 203 of the terminal device 506 performs user image acquisition from the user imaging device 107.

In step 802, the face detection unit 204 of the terminal device 506 performs face detection on the user image data acquired in step 801. Face detection is, for example, a method for obtaining a difference from a background image and determining the area based on the area or shape of the difference area, or arrangement of main components such as eyes, nose, and mouth prepared and learned in advance. Further, it is performed by a method of searching for the presence or absence of a region having a facial image feature pattern such as a difference between the forehead and the contrast of eyes using an in-image search. In another embodiment of the present invention, any method may be used.
If it is determined in step 802 that the face is detected (YES), the face detection unit 204 proceeds to step 803. If no face is detected (NO), the face detection unit 204 proceeds to step 808.

In step 803, the control unit 611 of the terminal device 506 turns off the no-monitorer flag and advances the process to step 804. For simplification of explanation, although not explicitly shown in FIG. 8, the no-monitorer flag is set to the initial state ON.
In step 804, the face feature amount calculation unit 608 of the terminal device 506 performs feature amount calculation on the face area obtained in step 802.
The feature quantities calculated here include, for example, image feature quantities such as the color of hair and skin, the shape and direction of the outline of the face, the size and shape of the main components such as eyes, nose, and mouth, and the arrangement relationship. However, in another embodiment of the present invention, any kind or number of feature quantities may be used.

In step 805, the person identifying unit 609 of the terminal device 506 performs feature amount collation. Specifically, the feature amount calculated in step 804 is sequentially collated (calculation of coincidence) for all the feature amounts of each person in the person data 700 recorded in advance in the person recording unit 610, Find the feature with the highest match. The person having this feature amount is set as the most similar person.

Here, the degree of coincidence is a numerical value indicating the closeness between images (image feature amounts), and the calculation method thereof is, for example, “Representation model for large-scale image collection” (see Non-Patent Document 1). Reference can be made to a paper such as Kaoru Tsujiike et al., Journal of the Japan Photography Society, Vol. 66, No. 1, P93-P101). It is assumed that the degree of coincidence in this example is higher as the value is smaller. For example, if the feature amount calculated in step 804 is 4.5, and the coincidence calculation method is a difference between feature amount values, the stored value of the feature amount cell 712 of three records 701 to 703 As a result of the collation, the person stored in the record 702 having a difference, that is, a matching degree of 0.4 becomes the most similar person. In the present invention, any method may be used for determining the collation order and calculating the degree of coincidence.

In step 806, the person identification unit 609 of the terminal device 506 performs a monitoring person's determination. In the determination, in the relationship between the degree of coincidence and the predetermined value (threshold value) for the most similar person obtained in step 805, the person is decided as a supervisor when the degree of coincidence is equal to or less than the threshold value. The threshold is set in advance. For example, if the threshold is 0.6, the degree of coincidence of the most similar person obtained in step 805 is 0.4, and therefore the person photographed by the user imaging device 107 is the person stored in the record 702. That is, it is determined to be a supervisor.
In the process of step 806, the person identification unit 609 proceeds to the process of step 807 if there is a supervisor (YES), and proceeds to the process of step 808 if there is no supervisor (NO).

In step 807, the control unit 611 of the terminal device 506 sets the person ID of the monitor obtained in step 806 as the monitor ID. For example, the stored value of the person ID cell 711 of the record 702 is set. After the process, the process returns to step 800. For simplification of explanation, although not explicitly shown in FIG. 8, the supervisor ID is not set in the initial state.
In step 808, the control unit 611 of the terminal device 506 turns on the no-monitorer flag, clears the supervisor ID, and returns to the process in step 800.

Next, the flow of processing in FIG. 8B will be described.
In step 810, the control unit 611 of the terminal device 506 waits for a predetermined time. The length of the waiting time is set in advance. After a predetermined time elapses, the process proceeds to step 811.
In step 811, the control unit 611 of the terminal device 506 determines whether or not the no-monitorer flag is ON. If the no-monitorer flag is OFF (NO), the process proceeds to step 812. If the no-monitorer flag is ON (YES), the process returns to step 810.

In step 812, the control unit 611 of the terminal device 506 determines a change in the supervisor ID. If the supervisor ID has changed from the previous determination (YES), the process proceeds to step 813, and the supervisor ID is set. If there is no change (NO), the process proceeds to step 815.
In step 813, the control unit 611 of the terminal device 506 acquires distribution conditions. From the person data 700 stored in the person recording unit 610, a record in which the supervisor ID matches the person ID is specified. Then, the distribution frame rate value and the image quality value are extracted from the distribution frame rate cell and the image quality cell of the matching record.

In step 814, the distribution instruction unit 200 of the terminal device 506 transmits an image distribution request to the monitoring imaging device. At this time, the requested distribution conditions, that is, the distribution frame rate and image quality are the values extracted in step 813.
In step 815, the monitoring image receiving unit 201 of the terminal device 506 waits for image reception. If an incoming image from the monitoring imaging apparatus is detected, the process proceeds to step 816.
In step 816, the monitoring image receiving unit 201 of the terminal device 506 receives a monitoring image from the monitoring imaging device.

In step 817, the screen display unit 202 of the terminal device 506 performs decompression of the compressed image and conversion of the image format on the received monitoring image data as necessary, and displays them on the screen. At that time, the distribution condition acquired in step 813 is referred to, and if the condition is high image quality, a large area is displayed, if the image quality is low, a small area is displayed, or if the frame rate is high, the screen is displayed. Display control is also performed in accordance with distribution conditions such as displaying at the top of the screen, and displaying at the bottom of the screen when the frame rate is low. This display control is performed in a coordinated manner so that the display positions do not overlap among the five processes executed in parallel. After the process, the process returns to step 811.

As described above, in one embodiment of the present invention, a system for obtaining a monitor for a monitoring image in real time and performing distribution while changing distribution conditions according to the monitor is provided. In addition, when a person does not exist in front of the terminal device or when a person exists but is not a supervisor, a system that does not perform distribution is provided. As a result, the waste of the network flow rate of the entire system can be reduced.

In the above description, for simplification of description, the configuration of five monitoring imaging devices and one terminal device is shown, but these configurations may be other than the above for the network.
In addition, although the configuration in which the monitoring video distribution source is an imaging device is shown, the configuration may be a recording device or a re-distribution device.
Moreover, although the process concerning a person identification was shown by the structure implemented on a terminal device, you may make it implement in a separate apparatus, for example, a server apparatus.
Furthermore, although the 1st form and 2nd form of this invention were shown as a separate Example, you may make it implement combining a 1st form and a 2nd form.

The monitoring system according to the embodiment of the present invention can reduce the network flow rate.

As mentioned above, although one embodiment of the present invention was described in detail, the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention.

-It can be applied to applications that want to reduce network flow rate by changing the image quality and frame rate of the distribution image based on the frequency of gaze at the image of the supervisor.

100: network 101-105: monitoring imaging device 106: terminal device 107: user imaging device 200: distribution instruction unit 201: monitoring image receiving unit 202: screen display unit 203: user image acquisition unit 204 : Face detection unit, 205: gaze determination unit, 206: determination result holding unit, 207: control unit, 300: distribution instruction data, 301 to 307: record, 310: frequency cell, 311: distribution frame rate cell, 312: image quality Cell, 506: terminal device, 608: face feature amount calculation unit, 609: person identification unit, 610: person recording unit, 611: control unit, 700: person data, 701 to 703: record, 710: record number cell, 711 : Person ID cell, 712: feature amount cell, 713 to 717: distribution frame rate cell, and 718 to 722: image quality cell.

Claims

A monitoring system including an imaging device, a user imaging device, and a terminal device,
The imaging apparatus has means for changing the image quality and frame rate of a distribution image,
The user imaging device photographs a supervisor of the terminal device,
The terminal device includes means for acquiring a gaze frequency for an image of a supervisor from a captured image of the user imaging device, means for determining an image quality or a frame rate of a distribution image based on the gaze frequency, and based on the determined result And a means for issuing a delivery instruction to the imaging apparatus.
A monitoring system including an imaging device, a user imaging device, and a terminal device,
The imaging apparatus has means for changing the image quality and frame rate of a distribution image,
The user imaging device photographs a supervisor of the terminal device,
The terminal device includes a means for displaying a delivery image from the imaging device, a means for personally identifying a supervisor from a photographed image of the user imaging device, and an image quality and a frame rate of the delivery image based on the personal identification result. And a means for issuing a distribution instruction to the imaging device based on the determined result.