JP2023093912A - Monitoring system - Google Patents

Abstract

To provide a monitoring system with which it is possible to conduct monitoring by an image in which consideration is taken for the privacy of an unvaccinated patient.SOLUTION: A monitoring system according to the present example comprises: an imaging device 10 for imaging a prescribed monitoring area; an image processing device 20 for determining, for each person included in the image captured by the imaging device 10, the necessity for anonymizing the person, and performing an anonymization process of processing an image portion of the person for whom anonymization is determined to be necessary; and a digital signage display device 30 for passengers and a display device 40 for facility operators on which the image having been processed by the image processing device 20 is displayed. The image processing device 20 refers to person data that includes the vaccination state of preregistered persons to confirm the vaccination state of the person included in the image, and identifies a person not confirmed to have been vaccinated (i.e., an unvaccinated parson or a person with unknown vaccination status), as the person who needs to be anonymized.SELECTED DRAWING: Figure 1

Description

The present invention relates to a surveillance system that displays an image of a predetermined surveillance area imaged by an imaging device on a display device.

In recent years, human detection technology and object recognition technology using deep learning and the like have been developed. These technologies make it possible to automatically recognize a person's physical characteristics such as gender and age, whether they are using a mask, and whether they are using nursing care products such as wheelchairs and white canes.

In order to utilize such person detection technology and object recognition technology, the installation of surveillance cameras is progressing for the purpose of safety and security of facility operation. On the other hand, it is necessary to consider the privacy of users at venues for vaccination against infectious diseases, which have been spreading in recent years, hospitals, commercial facilities, changing rooms and restrooms in airport facilities, or in the vicinity thereof. For this reason, there are places where surveillance cameras cannot be installed, even for the purpose of ensuring safety and security in facility operation and improving customer satisfaction. Therefore, conventionally, after performing object recognition and human behavior analysis, anonymized video that takes privacy into consideration is created and used for video monitoring.

Prior arts in the technical field related to the present invention include the following. For example, Patent Literature 1 discloses an invention that protects individual privacy by reducing or altering individual privacy information while maintaining a state in which it is possible to determine the identity of a person between multiple cameras.

JP 2006-287287 A

Surveillance footage that has too much anonymization of features such as vaccination status, whether masks are worn, personal features, and physical features is appropriately monitored for the 3 Cs (closed spaces, crowded places, close contact) as a countermeasure against infectious diseases. I was worried that I couldn't do it. In addition, the vaccination status of facility personnel is synonymous with showing the cleanliness of the facility itself and whether it is a safe and secure environment. It also had the advantage of improving satisfaction. However, the display of images that do not consider the intention of facility employees not to vaccinate poses a problem from the viewpoint of protecting the personal information of non-vaccinated persons.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a monitoring system capable of performing monitoring using images in consideration of the privacy of non-vaccinated persons.

In order to achieve the above object, a monitoring system according to one aspect of the present invention is configured as follows. That is, the monitoring system according to the present invention includes an image capturing device that captures an image of a predetermined monitoring area, and for each person included in the image captured by the image capturing device, determines whether or not the person is anonymized. An image processing device that performs anonymization processing for processing an image portion of a person determined to be necessary, and a display device that displays the image processed by the image processing device. Check the vaccination status of the person included in the image by referring to the person data including the vaccination status, and determine that the person who cannot be confirmed to have been vaccinated is the person who needs to be anonymized. It is characterized by

Here, in the monitoring system according to the present invention, the person data is associated with a face image of a person registered in advance or face feature data including the feature amount thereof, and the image processing device performs the For a person included in the image, it may be configured to identify the person through facial recognition processing using facial feature data and determine the need for anonymization based on the person data for the identified person.

Further, in the monitoring system according to the present invention, the person data is associated with a wearing article image corresponding to the affiliation of the person registered in advance or wearing article feature data including the feature amount thereof. identifying the person included in the image captured by the device by face recognition processing using face data when the affiliation of the person can be specified by the article recognition processing using the article characteristic data; It can be configured to determine the need for anonymization based on person data for the identified person.

Further, in the monitoring system according to the present invention, the image processing device can be configured to include persons who are included in images captured by the imaging device and who are not subject to person data as persons who need to be anonymized. .

Further, the monitoring system according to the present invention can be configured to display an alarm on the display device when the number of persons requiring anonymization within a predetermined range satisfies a predetermined value or more.

Further, the monitoring system according to the present invention may be configured to display an alarm on the display device when the condition satisfying the above condition continues for a predetermined time or longer.

ADVANTAGE OF THE INVENTION According to this invention, the monitoring system which can monitor by the image which considered the privacy of a non-vaccine person can be provided.

It is a figure showing an example of composition of a surveillance system concerning one embodiment of the present invention. 2 is a diagram showing an example of a flowchart of processing by the monitoring system of FIG. 1; FIG. FIG. 3 is a diagram showing a display example of an electronic signboard display device for passengers; It is a figure which shows the example of a display of the display apparatus for facility operators. It is a figure which shows the example of a display of the display apparatus for facility operators. It is a figure which shows the example of a display of the display apparatus for facility operators.

An embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows a configuration example of a monitoring system according to one embodiment of the present invention. The monitoring system of FIG. 1 includes an imaging device 10, an image processing device 20, an electronic signboard display device for passengers 30, a display device for facility operators 40, and an image recording device . These devices are connected to a network N, respectively. Although the monitoring system of this example is assumed to be applied to facilities such as stations and airports, it can be applied to various other facilities.

The imaging device 10 has a configuration of a camera and is arranged in various places. For example, as a surveillance camera, it is arranged so as to photograph a predetermined surveillance area. The imaging apparatus 10 can employ a configuration of a camera in which information is obtained by forming an image of incident light on an imaging element via a lens and a diaphragm. Examples of the imaging element here include a CCD (Charge-Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and the like. The imaging device 10 shoots an image at, for example, 3 frames per second (3 fps) or more, and the information is sent to the image processing device 20 and the image recording device 50 . A plurality of imaging devices 10 can be installed according to the situation such as the shape and size of the monitoring area.

The image processing device 20 has a control section 21 , a user interface section 22 , an image analysis section 23 , an anonymized image generation section 24 and an alarm generation section 25 .

The control unit 21 can exchange information with other devices through the network N, and controls the user interface unit 22 , the image analysis unit 23 , the anonymized image generation unit 24 and the alarm generation unit 25 . When the user inputs or operates the electronic signage display device 30 for passengers, the display device 40 for facility operators, or the like, the user interface unit 22 performs processing such as receiving information on the input or operation. The image analysis unit 23 analyzes the image captured by the imaging device 10, and performs processing for determining the necessity of anonymization for each person according to the vaccination status of each person included in the image. The anonymized video generation unit 24 performs processing for processing the image portion of the person determined to require anonymization in the image captured by the imaging device 10 . The alarm generation unit 25 performs processing for generating an alarm according to the congestion of unvaccinated persons.

The passenger electronic signboard display device 30 and the facility operator display device 40 are display devices capable of displaying the content processed by the image processing device 10 . The passenger electronic signage display device 30 is a display device provided to users of facilities such as stations and airports, and is installed at any place that can be visited by the users of the facilities. The facility operator display device 40 is a display device provided to persons concerned with the facility (operators and employees), and is installed in a place such as a monitoring room that can be used only by persons concerned with the facility. These display devices 30 and 40 perform display by a configuration such as a liquid crystal display (LCD), an organic EL (OEL) display, a touch panel, or the like. Also, the display devices 30 and 40 may be provided with operation buttons. The number of display devices 30 and 40 can be installed as required.

The image recording device 50 is a device that records images captured by the imaging device 10 . Further, the image recording device 50 may record images and results after processing by the image processing device 10 . As the image recording device 50, for example, HDD (Hard Disk Drive), SSD (Solid State Drive), DDS (Digital Data Storage), etc., can be applied as appropriate.

A network N is a line capable of data communication that connects each device. As the network N, a leased line, an intranet, an IP network such as the Internet, etc. can be applied regardless of the type of line.

FIG. 2 shows an example of a flow chart of processing by the monitoring system of this example. Here, it is assumed that person data in which persons concerned with facilities are registered in advance is stored in a database (not shown) that can be accessed from the image processing apparatus 20 . This person data includes data indicating the vaccination status of the person in addition to the name of the person. Also, the person data is associated with various feature data representing the features of the person. Examples of feature data include physical feature data indicating a person's age, gender, presence/absence of glasses, presence/absence of hearing aids, walking ability, etc.; clothing characteristic data necessary for the clothing recognition process for identifying the classification). The facial feature data includes, for example, a person's facial image or its feature amount. The article-of-wear characteristic data includes, for example, images of hats, uniforms, badges, armbands, belongings, etc., or their characteristic amounts, according to the department to which the wearer belongs.

The image processing device 20 analyzes the image captured by the imaging device 10 using the image analysis unit 23 . Specifically, the image analysis unit 23 first detects a person included in each frame of the image captured by the imaging device 10 (step S11). Next, the image analysis unit 23 extracts a feature amount related to each person detected from the image captured by the imaging device 10 (step S12). In this example, the feature amount of the face and the feature amount of the article to be worn are extracted as the feature amount of the person.

Next, the image analysis unit 23 performs wearing article recognition processing for collating the feature quantity of the worn article of the person detected from the image with the worn article feature data to determine whether the person is a facility related person belonging to the facility. is determined (step S13). For example, when clothing characteristic data whose degree of matching with the characteristic quantity of the clothing extracted from the image is equal to or greater than a predetermined value is found, the person is determined to be a person involved in the facility. On the other hand, if the wearing article characteristic data satisfying the above requirements is not found, the person is determined not to be a person related to the facility (that is, a facility user).

Next, the image analysis unit 23 performs face recognition processing for matching the facial feature amount extracted from the image with the facial feature data for the person determined to be the person involved in the facility, thereby identifying the person. A vaccination status is determined based on the person data associated with the facial feature data (step S14). For example, when face feature data whose degree of matching with a face feature quantity extracted from an image is equal to or greater than a predetermined value is found, the person data associated with the face feature quantity is referred to determine whether the person has been vaccinated or not. judge. On the other hand, if facial feature data satisfying the above requirements is not found, it is determined that vaccination is unknown. In addition, the image analysis unit 23 determines that vaccination is unknown for a person determined not to be a person related to the facility (that is, a facility user) without performing face recognition processing.

Next, the image analysis unit 23 performs a process of recording in the memory the determination result of steps S13 and S14, that is, the determination result of whether or not the person is related to the facility and the determination result of the vaccination status (step S15). The above processing is repeatedly executed for all persons included in the image captured by the imaging device 10 (step S16).

Next, the anonymized image generation unit 24 selects a person who has not been confirmed to have been vaccinated by the analysis in the image analysis unit 23, that is, a person who has not been vaccinated or whose vaccination is unknown. Assuming that the person is a person, the image portion of the person is processed and anonymized (step S17). For example, in face anonymization, a face image or face part image of a target person is compared with a plurality of types of pre-registered abstract face images or abstract face part images (for example, feature amount comparison), The abstract face image or the abstract face part image having the highest degree of similarity is replaced and synthesized. Similarly, anonymization of the body and clothes other than the face may be performed. Alternatively, one or a plurality of avatar images may be prepared in association with age and sex, and replaced with an avatar image corresponding to the age and sex of the target person.

Next, the alarm generation unit 25 determines the necessity of an alarm according to the degree of crowding of persons who need to be anonymized (that is, persons who have not been vaccinated or whose vaccination is unknown), and when it is determined that an alarm is necessary (step S18). For example, it is determined that an alarm is necessary when the number of persons requiring anonymization within a predetermined range satisfies the condition that the number is equal to or greater than a predetermined value. Alternatively, it is determined that an alarm is necessary when the condition satisfying the above condition continues for a predetermined time or longer. Also, the behavior of a person who needs to be anonymized may be analyzed by detecting the skeleton or the like, and used for the above determination.

The anonymized image generated by the anonymized image generator 24 is transmitted to the passenger electronic signage display device 30 and the facility operator display device 40 and displayed on their screens. Further, when an alarm is issued by the alarm generator 25, the data is transmitted to the passenger electronic signage display device 30 and the facility operator display device 40 and displayed on their screens.

FIG. 3 shows a display example of the electronic signboard display device 30 for passengers. The screen of the electronic signboard display device 30 for passengers in this example includes a display section 31 for vaccination status at required points in the facility, a display section 32 for a map of the facility, and a camera (imaging device 10) in the facility. and a display unit 33 for displaying an anonymized image obtained by subjecting the imaged image to anonymization processing.

On the display unit 31, for example, for each of the three gates A, B, and C installed in the facility, the ratio of vaccinated facility personnel and non-vaccinated facility personnel in the vicinity is displayed in a pie chart. Is displayed. This ratio is obtained by analyzing the image captured by the imaging device 10 installed in the vicinity of the area by the image processing device 10, can be calculated by specifying

For example, the cameras to be displayed by the display unit 33 are automatically switched at regular intervals. Further, the camera to be displayed by the display unit 33 may be selected by the facility user by operating the electronic signboard display device 30 for passengers. For example, when a facility user touches a map displayed on the display unit 32, an anonymized image of a camera installed at a location within the facility corresponding to the touch position is acquired and displayed on the display unit 33. may

4, 5, and 6 show display examples of the facility operator display device 40. FIG. The screen of the facility operator display device 40 in this example includes a display section 41 for a map of the facility and vaccination status at required locations in the facility, and an image captured by one of the cameras (imaging device 10) in the facility or An anonymized image display unit 42, an operation unit 43 for receiving a selection operation of a camera to be displayed by the display unit 42 from a facility official, an alarm occurrence status display unit 44, and an operation for confirming details of an alarm. It also has an operation unit 45 that receives information from facility personnel.

As shown in FIG. 4, the facility operator's display device 40 normally uses an area that occupies most of the screen as a display section 41 for the map of the facility and the vaccination status at required points in the facility. 5 or 6, the display unit 42 is switched to the image captured by the camera (imaging device 10) or the anonymized image. FIG. 5 shows an example of a format in which an image captured by a camera is displayed as it is on the display unit 42, and FIG. 6 shows an example of a format in which an image captured by a camera is anonymized and displayed on the display unit 42. is. Which display format to use may be determined in advance, or may be dynamically switched according to the authority of the user of the facility operator display device 40 or the like.

The display unit 44 displays the status of alarm occurrences in the facility in chronological order. Further, when the operation unit 45 is operated while any alarm on the display unit 44 is selected, the details of the selected alarm are further displayed. At this time, the image at the time of the alarm or the anonymized image may be read from the image recording device 50 and displayed on the display unit 42 .

As described above, the monitoring system of this example determines the necessity of anonymization for each person included in the image captured by the imaging device 10 that captures an image of a predetermined monitoring area and the person included in the image captured by the imaging device 10. , an image processing device 20 that performs anonymization processing to process an image portion of a person determined to require anonymization, an electronic signboard display device 30 for passengers that displays the image processed by the image processing device 20, and a facility operator The image processing device 20 refers to person data including the vaccination status of the person registered in advance, confirms the vaccination status of the person included in the image, and confirms that the person has been vaccinated. A person for whom this could not be confirmed (that is, a person who has not been vaccinated or whose vaccination is unknown) is determined to be a person who needs to be anonymized.

Therefore, according to the monitoring system of this example, since the vaccination status can be presented to the facility user by the electronic signboard display device 30 for passengers, if the vaccination status of the facility personnel is low, the virus infection from the facility personnel can be prevented. It is possible to inform the facility users that the facility is safe and secure with a low possibility of On the other hand, by anonymizing persons who have not been vaccinated or whose vaccination is unknown, respecting the individual's privacy and intention of not being vaccinated, it is possible to monitor using images that respect the privacy of those who have not been vaccinated. becomes.

In the monitoring system of this example, the image processing device 20 recognizes the person included in the image captured by the imaging device 10 by performing the clothing recognition process using the clothing characteristic data. person) can be identified, the person is identified by face recognition processing using face data, and the need for anonymization is determined based on the person data for the identified person. there is With such a configuration, it is possible to omit the face recognition processing for a person who is found not to be a person related to the facility by the wearing article recognition processing, so that the vaccination status can be determined efficiently.

Here, in this example, the anonymization process is performed to process the image portion of the unvaccinated person, but the image portion of the vaccinated person may also be processed. In other words, for example, the facial images of persons involved in the facility are photographed and stored without anti-infection products such as masks and face guards, and the facial images of vaccinated persons are used as the facial images without anti-infection products. be replaced with This makes it easier for the person concerned in the facility to identify the person in the image when performing the monitoring work.

In addition, in this example, all facility users are anonymized as persons whose vaccination status is unknown. You may exclude it from the object of anonymization processing for the purpose. This makes it easier to recognize the person in need of support, so that facility personnel can quickly go to that person.

Although the embodiments of the present invention have been described above, these embodiments are merely examples and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and various modifications such as omission and replacement can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in this specification and the like, and are included in the scope of the invention described in the claims and equivalents thereof.

In addition, the present invention can be provided not only as devices such as those mentioned in the above description and systems configured with these devices, but also as methods executed by these devices and functions of these devices by a processor. It is also possible to provide a program for implementation, a storage medium storing such a program in a computer-readable manner, and the like.

INDUSTRIAL APPLICABILITY The present invention can be used in a surveillance system that displays an image of a predetermined surveillance area captured by an imaging device on a display device.

10: Imaging device 20: Image processing device 21: Control unit 22: User interface unit 23: Image analysis unit 24: Anonymized image generation unit 25: Alarm generation unit 30: Electronic signboard display device for passengers , 40: Display device for facility operator, 50: Image recording device

Claims (6)

an imaging device for imaging a predetermined monitoring area;
An image processing device that performs anonymization processing for determining the necessity of anonymization of each person included in the image captured by the imaging device and processing the image portion of the person determined to require anonymization. and,
A display device for displaying an image processed by the image processing device,
The image processing apparatus refers to person data including the vaccination status of a person registered in advance, confirms the vaccination status of the person included in the image, and confirms the vaccination status of the person who has not been confirmed to have been vaccinated. is a person who needs to be anonymized. The monitoring system of claim 1, wherein
The person data is associated with a face image of a person registered in advance or face feature data including a feature amount thereof,
The image processing device identifies a person included in the image captured by the imaging device by face recognition processing using the facial feature data, and anonymizes the identified person based on the person data. A surveillance system characterized by determining need. In the monitoring system of claim 2,
The person data is associated with a wearing article image registered in advance according to the affiliation of the person or wearing article feature data including a feature amount thereof,
The image processing device uses the face data when the affiliation of the person included in the image captured by the imaging device can be identified by the clothing item recognition processing using the clothing item feature data. A surveillance system that identifies a person by face recognition processing and determines the need for anonymization based on person data for the identified person. In the monitoring system according to any one of claims 1 to 3,
The surveillance system, wherein the image processing device includes persons included in the image captured by the image pickup device, who are not subject to the person data, as persons who need to be anonymized. In the monitoring system according to any one of claims 1 to 4,
A monitoring system characterized by displaying an alarm on the display device when the number of persons requiring anonymization within a predetermined range satisfies a predetermined value or more. In the monitoring system of claim 5,
A monitoring system, wherein an alarm is displayed on the display device when the condition satisfying the condition continues for a predetermined time or longer.

Images