JP2023063956A - Image processing system, image processing method, image processing program, and image processing device - Google Patents

Abstract

To provide an image processing system for identifying a sanitary mask worn by a user and determining whether or not the sanitary mask satisfies given conditions in light of the fact that people generally wear sanitary masks in public spaces.SOLUTION: The present invention relates to an image processing system for processing a captured image of a face of a user wearing a sanitary mask, the image processing system comprising a detection unit 13 configured to detect a mask-presence area with an image of a sanitary mask in the captured image, and an acquisition unit 16 configured to acquire sanitary mask information on the sanitary mask based on a mask-presence area A3.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image processing system, an image processing method, an image processing program, and an image processing apparatus.

For example, the authentication system disclosed in Patent Literature 1 is configured to authenticate the user's eyes without authenticating the user's entire face when the user is wearing a mask.

JP 2020-166597 A

By the way, due to the worldwide spread of the coronavirus called "COVID-19" in 2020 AD, it has become common for people to wear sanitary masks in public places. Various sanitary masks such as sanitary masks with different functions and unique sanitary masks have come to exist. Also, the method of wearing a sanitary mask is different for each user. As a result, there is a problem that the shape and color of the sanitary mask and the method of wearing the sanitary mask are incompatible with the rules and regulations in workplaces, schools, facilities, and the like. However, in the prior art, there is nothing to distinguish the suitability of sanitary masks, and it is necessary to deal with the above problems.

An object of the present invention is to provide an image processing system that can contribute to proper wearing of a sanitary mask.

A characteristic configuration of an image processing system for achieving the above object is an image processing system that processes a captured image of the face of a user wearing a sanitary mask, wherein the sanitary mask is captured from the captured image. The present invention is provided with a detection unit that detects a mask existence area and an acquisition unit that acquires sanitary mask information related to the sanitary mask based on the mask existence area.

According to the above characteristic configuration, by capturing an image of the user wearing the sanitary mask, the detection unit detects the presence area of the sanitary mask, and the acquisition unit acquires information about the sanitary mask. Therefore, with the above-described characteristic configuration, it is possible to use the sanitary mask information acquired from the captured image to perform various measures for proper wearing of the sanitary mask. For example, it is possible to determine whether or not the user is wearing an appropriate sanitary mask, determine whether or not the wearing style of the sanitary mask is appropriate, and the like.

Another characteristic configuration of the image processing system according to the present invention is a storage unit that stores condition information indicating a predetermined condition regarding the sanitary mask, and the sanitary mask information and the condition information are collated to determine whether the sanitary mask is worn by the user. and a determination unit that determines whether the sanitary mask satisfies the predetermined condition.

With the above characteristic configuration, the storage unit is configured to store the condition information, and the condition information indicates the predetermined condition regarding the sanitary mask. Condition information that conforms to rules and regulations can be stored in advance in the storage unit. Then, the determination unit compares the sanitary mask information with the condition information to determine whether the sanitary mask satisfies a predetermined condition. Therefore, it is possible to take various measures for proper wearing of the sanitary mask based on the determination result. For example, it is possible to encourage users who do not meet predetermined conditions to properly wear sanitary masks in accordance with the rules and regulations of workplaces, schools, and other facilities.

Another characteristic configuration of the image processing system according to the present invention is further provided with a user type acquisition unit that acquires user information indicating the type of user, and the storage unit stores information corresponding to the type of user. It is configured to store a plurality of pieces of condition information, and the determination unit is configured to determine whether or not the sanitary mask worn by the user satisfies the predetermined condition corresponding to the type of the user. in that it is

For example, if the facility is a hospital, the users may be outpatients, patients, doctors, nurses, clerks, etc., and each user may wear a different type of sanitary mask. For this reason, it is desirable that the predetermined conditions regarding the sanitary mask are classified according to the type of user. According to the above characteristic configuration, the condition information is stored in the storage unit according to the user type, and the determination unit reads out the condition information from the storage unit according to the user type. can flexibly determine the suitability of a sanitary mask.

Another characteristic configuration of the image processing system according to the present invention further includes a region information acquisition unit that acquires a region where the captured image was captured, and the storage unit stores a plurality of the condition information corresponding to the region. and the determination unit is configured to determine whether or not the sanitary mask worn by the user satisfies the predetermined condition corresponding to the region.

When COVID-19 spread in the year 2020, lockdowns were put in place around the world. In areas where city blockades are being implemented, the predetermined conditions may be set more severely than in other areas. According to the above characteristic configuration, determination is made based on the predetermined conditions corresponding to the area where the captured image was taken, so appropriate determination is made according to the situation of the area.

Another characteristic configuration of the image processing system according to the present invention is that the storage unit is configured to store a plurality of the condition information corresponding to time periods, and the determination unit stores the condition information worn by the user. The point is that it is configured to determine whether or not the sanitary mask satisfies the predetermined condition corresponding to the time period.

When COVID-19 spread in 2020 AD, a state of emergency was declared in Japan, and during the state of emergency, usage of facilities such as stores was restricted according to the time of day. Therefore, the predetermined condition may be changed according to the time period. According to the above characteristic configuration, the condition information is stored in the storage unit according to the time period, and the determination unit reads out the condition information from the storage unit according to the time period. It is possible to judge the suitability of the mask.

Another characteristic configuration of the image processing system according to the present invention is further provided with an input unit for the user to input a matter, and the storage unit stores a plurality of the condition information corresponding to the matter. and the determination unit is configured to determine whether or not the sanitary mask worn by the user satisfies the predetermined condition corresponding to the requirement.

For example, if the facility is a financial institution, users may use the facility for various purposes such as opening and changing accounts, consulting on investments and loans, and the like. Depending on the business, multiple people may gather in an enclosed space, and in this case, it is necessary to pay attention to conditions such as wearing sanitary masks. According to the above characteristic configuration, the condition information is stored in the storage unit according to the task, and the determination unit reads out the condition information from the storage unit according to the task. It is possible to judge the suitability of the mask.

Another characteristic configuration of the image processing system according to the present invention further includes a notification unit that controls a notification device to notify the user, and the notification unit sends the user wearing the sanitary mask to the predetermined It is characterized in that it notifies whether or not a condition is satisfied and, when the predetermined condition is not satisfied, notifies the user of the reason why the predetermined condition is not satisfied.

With the above-described characteristic configuration, the notification unit is configured to notify the user whether or not the predetermined condition is satisfied, and if the predetermined condition is not satisfied, the reason is also notified together, thereby providing hygiene to the user. It is possible to encourage proper wearing of masks.

Another characteristic configuration of the image processing system according to the present invention is that the detection unit is configured to be capable of detecting first face information related to the user's face from the captured image, and from the first face information, the use A first user identification unit for identifying a user is further provided, and the determination unit is configured to be able to store the number of times the predetermined condition is not satisfied for each user in the storage unit.

Even if the user tries to wear the sanitary mask appropriately, it may not always be worn properly due to the user's daily habits. According to the above characteristic configuration, since the storage unit can store the number of times that the predetermined condition is not satisfied for each user, it is possible to make the user aware of proper wearing of the sanitary mask. In addition, by recording the fact that the prescribed conditions were not met, it is possible to analyze trends in how users wear sanitary masks, and to formulate measures for users to wear sanitary masks appropriately. easier.

Another characteristic configuration of the image processing system according to the present invention is that the acquisition unit acquires the color and shape of the sanitary mask worn by the user as the sanitary mask information based on the mask existing area. The point is that it is configured to acquire at least one.

For example, in schools, the color and shape of sanitary masks may be specified, and in workplaces, the colors of sanitary masks may be restricted so that they are not flashy. According to the above characteristic configuration, since the acquisition unit acquires at least one of the color and shape of the sanitary mask, it is possible to determine whether the user is wearing a proper sanitary mask according to rules and regulations. It is possible to construct an image processing system for determination.

Another characteristic configuration of the image processing system according to the present invention is that the acquisition unit obtains at least one of the position and angle of the sanitary mask on the user's face as the sanitary mask information based on the mask existence area. in that it is configured to obtain

If the sanitary mask covers only the part below the user's nose, or if the sanitary mask is biased to either the left or right side of the user's face, the sanitary mask cannot fulfill its original function. According to the above characteristic configuration, since the acquisition unit is configured to acquire at least one of the position and angle of the sanitary mask on the user's face, whether the user is wearing the sanitary mask in an appropriate manner. It is possible to construct an image processing system that determines whether or not

Another characteristic configuration of the image processing system according to the present invention is that the acquisition unit is configured to acquire the type of the sanitary mask as the sanitary mask information based on the mask existing area. .

For example, sanitary masks are classified into cloth, nonwoven fabric, urethane, and the like in terms of material. In addition to the material, the types of sanitary masks are classified into various types such as pleated sanitary masks, three-dimensional sanitary masks, and sanitary masks conforming to various standards of the US National Institute for Occupational Safety and Health. According to the above characteristic configuration, since the acquisition unit is configured to acquire the type of sanitary mask, whether the user is wearing an appropriate sanitary mask in a facility that requires strict sanitary management such as a medical site, for example. It is possible to construct an image processing system that determines whether or not

Another characteristic configuration of the image processing system according to the present invention is configured such that the acquisition unit acquires personal identification information written on the sanitary mask as the sanitary mask information based on the mask existing area. in the point.

If a barcode, two-dimensional code, color code, or the like is written on the sanitary mask, the sanitary mask can also be used as an employee ID card. According to the above characteristic configuration, focusing on the fact that personal identification information can be attached to the sanitary mask, the acquisition unit acquires the personal identification information written on the sanitary mask, so there is no separate employee ID card or the like. Personal authentication with a sanitary mask becomes possible.

Another characteristic configuration of the image processing system according to the present invention is that the detection unit is configured to be able to detect first face information related to the user's face from the captured image, and based on the personal identification information, A second user identification unit that identifies second face information related to the face of the user, and a face matching unit that performs face authentication of the user by matching the first face information and the second face information. and are further provided.

According to the above characteristic configuration, the user's face information is stored as the second face information in association with the personal identification information, and the user's face information can be specified from the personal identification information. Also, the face information of the user is detected as the first face information from the captured image. Therefore, the user's face can be authenticated by the face collation unit collating the first face information and the second face information. This configuration enables two-factor authentication of authentication based on the sanitary mask (possession authentication) and authentication based on the user's face information (biometric authentication).

Another characteristic configuration of the image processing system according to the present invention further includes a notification unit that controls a notification device to notify the user, and the notification unit informs the user whether or not the face authentication is successful. When the face matching unit fails in the face authentication, the user is notified to remove the sanitary mask, and the detection unit notifies that the user has removed the sanitary mask. third face information about the face of the user can be detected from the captured image in the state, and the face matching unit compares the second face information with the third face information to determine the The point is that it is configured to perform face authentication.

When the user is wearing a sanitary mask, the face matching unit may match the portion of the user's face above the sanitary mask (eyes, forehead, hair). In some cases, there is a possibility that the accuracy of face authentication may be degraded. According to the above characteristic configuration, when the face matching section fails in face recognition, the notification section notifies the user to remove the sanitary mask. As a result, it is possible to obtain a captured image of the user without the mask, and the face matching unit can match the third face information of the user without the mask with the second face information. In other words, matching can be performed over the entire area of the user's face. As a result, the face matching unit can perform highly accurate face authentication.

Another characteristic configuration of the image processing system according to the present invention is that the notification unit notifies the user to wear the sanitary mask when the face authentication is successful.

The above-described characteristic configuration suppresses the possibility that the user forgets to put on the sanitary mask again after face authentication is performed while the user has removed the sanitary mask.

Another characteristic configuration of the image processing system according to the present invention is that it further includes a re-wear confirmation unit that determines whether or not the user is wearing the sanitary mask after the face authentication is successful.

With the above-described characteristic configuration, it is determined whether or not the sanitary mask has been removed and the mask has been put back on after face authentication has been performed. For example, it is possible to prohibit a user who has not re-worn from entering the room, or to issue a notification to encourage the user to re-worn.

The characteristic configuration of the image processing system of the present invention can also be applied to the image processing method. A feature configuration of an image processing method for achieving the above object is an image processing method for processing a captured image of the face of a user wearing a sanitary mask, wherein a computer processes the sanitary mask from the captured image. The present invention comprises a detection step of detecting a photographed mask existence area, and an obtaining step of obtaining, in a computer, sanitary mask information regarding the sanitary mask based on the mask existence area.

According to the above characteristic configuration, the detection step detects the presence area of the sanitary mask by imaging the user wearing the sanitary mask, and the obtaining step acquires information about the sanitary mask. Therefore, with the above-described characteristic configuration, it is possible to use the sanitary mask information acquired from the captured image to perform various measures for proper wearing of the sanitary mask. For example, it is possible to determine whether or not the user is wearing an appropriate sanitary mask, determine whether or not the wearing style of the sanitary mask is appropriate, and the like.

The characteristic configuration of the image processing system of the present invention can also be applied to an image processing program. A feature configuration of an image processing program for achieving the above object is an image processing program for causing a computer to process a captured image of the face of a user wearing a sanitary mask, wherein the sanitary mask is captured from the captured image. The computer is caused to execute a detection function of detecting a mask existence area to be detected and an acquisition function of acquiring sanitary mask information related to the sanitary mask based on the mask existence area.

According to the above characteristic configuration, by capturing an image of the user wearing the sanitary mask, the detection function detects the presence area of the sanitary mask, and the acquisition function acquires information about the sanitary mask. Therefore, with the above-described characteristic configuration, it is possible to use the sanitary mask information acquired from the captured image to perform various measures for proper wearing of the sanitary mask. For example, it is possible to determine whether or not the user is wearing an appropriate sanitary mask, determine whether or not the wearing style of the sanitary mask is appropriate, and the like.

The characteristic configuration of the image processing system of the present invention can also be applied to an image processing apparatus. A characteristic configuration of an image processing device for achieving the above object is an image processing device that processes a captured image of the face of a user wearing a sanitary mask, wherein the sanitary mask is captured from the captured image. The present invention is provided with a detection unit that detects a mask existence area and an acquisition unit that acquires sanitary mask information related to the sanitary mask based on the mask existence area.

According to the above characteristic configuration, by capturing an image of the user wearing the sanitary mask, the detection unit detects the presence area of the sanitary mask, and the acquisition unit acquires information about the sanitary mask. Therefore, with the above-described characteristic configuration, it is possible to use the sanitary mask information acquired from the captured image to perform various measures for proper wearing of the sanitary mask. For example, it is possible to determine whether or not the user is wearing an appropriate sanitary mask, determine whether or not the wearing style of the sanitary mask is appropriate, and the like.

The above and other configurations, features, methods and effects of the present invention will become clearer with reference to the following embodiments and drawings.

1 is a block diagram showing an image processing system for judging suitability of a sanitary mask; FIG. It is a flowchart figure which shows the propriety judgment of a sanitary mask. 1 is a block diagram showing an image processing system that performs personal authentication based on a sanitary mask; FIG. FIG. 3 is a flow chart diagram showing personal authentication based on a sanitary mask. FIG. 10 is a diagram showing room entry determination based on a change in the number of inter-eye pixels; FIG. 10 is a diagram showing room entry determination based on a change in the number of inter-eye pixels;

Preferred embodiments of the image processing system will be described in detail below with reference to the accompanying drawings. In the following embodiments, a case where the facility is a hospital will be mainly described as an example, but the facility is not limited to a hospital. The image processing system of this embodiment can be used in educational facilities, nursery schools, welfare facilities, financial institutions, post offices, restaurants, retail stores, office buildings, stations such as railways and buses, airports, accommodation facilities, factories, construction sites, It can be used in various facilities such as electric power facilities, water supply facilities, distribution facilities, agriculture and forestry facilities, fisheries facilities, research facilities, military facilities, government facilities, event venues, entertainment facilities, and sports facilities.

[Image processing system for judging suitability of sanitary masks]
FIG. 1 is a block diagram showing an image processing system for judging suitability of a sanitary mask. The image processing system of the embodiment shown in FIG. The camera 1, the subject input unit 2, the local information acquisition unit 3, and the notification device 4 may be configured to be integrated into one device, such as a tablet computer.

The camera 1 is configured to capture color video and images a user wearing a sanitary mask. Camera 1 may be configured to capture infrared video in addition to color video.

The message input unit 2 is used by a user who visits the facility to input a message, and is operated by the user, for example, on a touch panel monitor. When the matter input unit 2 is a soft button of a touch panel monitor, for example, a plurality of soft buttons are displayed for each matter on the touch panel monitor, and the user touches the soft button.

The area information acquisition unit 3 acquires information about the area where the camera 1 is installed. The regional information acquisition unit 3 may be, for example, a GNSS (Global Satellite Navigation System) positioning device, a beacon receiver, or a LAN access point that obtains position information and regional information. It may be acquired. Also, the area information acquisition unit 3 may be configured to acquire area-related information based on a human operation. For example, a regional setting soft button is displayed on a touch panel monitor of a tablet computer, and the regional information acquisition unit 3 acquires the regional information by operating the soft button and specifying regional information. It may be a configuration.

The notification device 4 may be, for example, a speaker that outputs a buzzer or voice guidance, a monitor that outputs visual information, or a device that outputs both voice information and visual information. good. When the matter input unit 2 is composed of a touch panel monitor, the touch panel monitor may also be used as the notification device 4 . The notification device 4 may be a device that sends an e-mail.

The image processing system of the embodiment shown in FIG. identification unit”), a mask information acquisition unit 16, a condition information acquisition unit 17, a determination unit 18, and a notification unit 19 are provided. The functional units (storage unit 11 to notification unit 19) described above are program modules built in an image processing server capable of real-time processing related to image analysis processing, and are configured to operate in a cloud environment. The video input unit 12 is communicably connected to the camera 1, the message input unit 2 and the area information acquisition unit 3 are communicably connected to the condition information acquisition unit 17, and the notification device 4 is communicably connected to the notification unit 19. It is

The storage unit 11 stores condition information indicating predetermined conditions regarding the sanitary mask in a storage medium (for example, hard disk or flash memory). Predetermined conditions differ depending on, for example, hospitals, schools, offices, event venues, etc., and the administrator or operator of the image processing system of the present embodiment appropriately sets condition information for each facility to be used, and stores the condition information in the storage unit 11. .

The storage unit 11 also stores user information indicating the user and the type of the user. For example, when the facility is a hospital, the user information of medical staff is stored in the storage unit 11 by classifying it into categories such as doctors, nurses, counselors, and clerks. The user information includes face information of the user. Face information is information for specifying a person, and is information obtained by digitizing a face image or features (intervals and positional relationships of eyes, nose, mouth, etc.) from the face image. User information may include personal identification information such as a name.

A camera 1 captures an image of a user wearing a sanitary mask, and the captured image captured by the camera 1 is captured by a video input unit 12 as captured image data. In the example shown in FIG. 1, an imaging area A1 surrounded by a dashed line is captured as captured image data. The captured image data is transmitted from the video input unit 12 to the detection unit 13 .

The detection section 13 includes a face detection section 13A and a mask detection section 13B. The face detection unit 13A detects a face information area A2 in which the user's face exists, from the imaging area A1 captured by the video input unit 12. FIG. The mask detection unit 13B detects the mask existing area A3 where the sanitary mask exists, for example, by extracting a portion different from the skin color from the face information area A2.

The face detection unit 13A transmits information (position, shape, etc.) regarding the detected face information area A2 to the user type acquisition unit 14 and the user identification unit 15, respectively. The mask detection unit 13B transmits information (position, shape, etc.) regarding the detected mask existence area A3 to the mask information acquisition unit 16. FIG. The mask detection unit 13B corresponds to the "detection function" of the present invention.

The user identification unit 15 extracts the user's face information based on the face information area A2 detected by the face detection unit 13A. Then, the user identification unit 15 searches for user information stored in the storage unit 11 based on the extracted face information. The user information includes the face information of the user, and the user identification unit 15 searches for the face information in the user information. Then, when the user's face information is found, the user identification unit 15 identifies the user based on the user information. At this time, the user type acquisition unit 14 acquires the user type based on the user information.

It should be noted that, for example, face information of an outpatient of a hospital is often not stored in the storage unit 11, and the user identification unit 15 may fail to identify the user. In this case, the user type acquiring unit 14 acquires, for example, the type of "other" (a type prepared in case the user fails to be specified) as the type of the user.

The user identification unit 15 transmits user identification information to the determination unit 18 . Also, the user type acquisition unit 14 transmits information about the type of user to the condition information acquisition unit 17 .

The mask information acquisition unit 16 acquires sanitary mask information on the sanitary mask based on the mask existence area A3 where the sanitary mask exists. Specifically, the mask information acquisition unit 16 acquires sanitary mask information by image analysis based on the mask presence area A3 detected by the mask detection unit 13B. Sanitary mask information includes the color and shape of the sanitary mask, the position and angle of the sanitary mask on the user's face, the type of sanitary mask, and the like.

The color of the sanitary mask includes the color, pattern, pattern, etc. of the sanitary mask. The shape of the sanitary mask includes the outer shape, three-dimensional shape, and the like of the sanitary mask.

The type of sanitary mask includes the material of the sanitary mask (cloth, non-woven fabric, urethane, etc.), the standard of the sanitary mask (standard of the US National Institute for Occupational Safety and Health, etc.), and the like.

The mask information acquisition unit 16 transmits the sanitary mask information to the determination unit 18 . The mask information acquisition unit 16 is configured to be able to acquire the type of sanitary mask based on the color information and shape information of the sanitary mask, the surface temperature of the sanitary mask, and the like. The surface temperature of the sanitary mask can be obtained from infrared information, for example. If the camera 1 has a function of capturing an infrared image, the image input unit 12, the mask detection unit 13B, and the mask information acquisition unit 16 can acquire the surface temperature of the sanitary mask. The mask information acquisition unit 16 corresponds to the "acquisition unit" and "acquisition function" of the present invention.

The condition information acquisition unit 17 obtains the type of user, the matter input by the matter input unit 2, the area information obtained by the area information acquisition unit 3, and the sanitary mask information obtained by the mask information acquisition unit 16. The condition information is read out from the storage unit 11 based on at least one of the time zone in the timing and the timing. The condition information acquisition unit 17 then transmits the condition information to the determination unit 18 .

The determination unit 18 collates the condition information received from the condition information acquisition unit 17 and the sanitary mask information received from the mask information acquisition unit 16 to determine whether the sanitary mask worn by the user satisfies a predetermined condition. judge. The determination result of the determination unit 18 is transmitted to the notification unit 19 .

The notification unit 19 controls the notification device 4 to notify the user whether or not the sanitary mask satisfies a predetermined condition. When the sanitary mask satisfies the predetermined condition, the notification unit 19 notifies the user via the notification device 4 that the sanitary mask satisfies the predetermined condition. When the sanitary mask does not satisfy the predetermined conditions, the notification unit 19 notifies the user of the fact that the sanitary mask does not satisfy the predetermined conditions and the reason thereof via the notification device 4 .

User information is sent from the user identification unit 15 to the determination unit 18 . When the sanitary mask does not satisfy the predetermined condition, the determination unit 18 stores the number of times the predetermined condition was not satisfied in the storage unit 11 for each user. Therefore, it is possible to make the user aware of proper wearing of the sanitary mask. In addition, by storing in the storage unit 11 that the predetermined condition is not satisfied, it is possible to analyze the tendency of the user's wearing method of the sanitary mask, and the user can wear the sanitary mask appropriately. Countermeasures become easier.

It should be noted that the determination unit 18 may not store the number of times the predetermined condition is not satisfied in the storage unit 11 when the user identification unit 15 fails to identify the user. Alternatively, when the user identification unit 15 fails to identify the user, the determination unit 18 simply records the date and time of determination that the predetermined condition was not satisfied in the storage unit 11 regardless of the user. Also good.

The condition information indicating the predetermined condition regarding the sanitary mask is appropriately set according to the user's type, area, time zone, business, and the like. Therefore, a plurality of pieces of condition information are stored in the storage unit 11 . For example, the condition information may be configured as a two-dimensional or higher array table according to items such as user type, area, time zone, and business. It should be noted that the condition information does not have to consist of all of the user type, region, time zone, and task, and may consist of at least one of facility, user type, region, time zone, and task. Also good.

For example, in hospitals, doctors and nurses may be required to wear sanitary masks conforming to the N95 standard of the US National Institute for Occupational Safety and Health. In addition, it is conceivable that hospital clerks may be prohibited from wearing extravagant or unusual sanitary masks by specifying the color, shape, etc. of their sanitary masks.

In this case, for example, a manager or a system operator stores user information such as doctors, nurses, clerks, etc. working in a hospital in advance in the storage unit 11, and sets user types as "doctor", "nurse", It is possible to store a plurality of pieces of condition information in the storage unit 11 for each type of user by classifying them into "clerk" and "others". At this time, the condition information when the user type is "doctor" and "nurse" is set to "conform to the N95 standard", and the condition when the user type is "clerk" The information may be set as 'specified color and shape', and the condition information when the user's type is 'others' may be set as 'commercially available sanitary masks are acceptable'.

When the user identification unit 15 identifies users such as doctors, nurses, and clerks who work at the hospital, the condition information acquisition unit 17 identifies “doctors,” “nurses,” “clerks,” and the like. Read condition information according to the type of user. On the other hand, users who visit hospitals are often not stored in the user information of the storage unit 11 . In this case, since the user identification unit 15 cannot identify the user from the user information in the storage unit 11, the user type acquisition unit 14 acquires the type of "other", and the condition information acquisition unit 17 acquires the type of "other". The type condition information is read from the storage unit 11 .

Then, the determination unit 18 determines whether or not the doctor or nurse is wearing a sanitary mask conforming to the N95 standard in an appropriate manner. The determination unit 18 also determines whether or not the clerk is wearing the designated sanitary mask in an appropriate manner. Furthermore, the determination unit 18 determines whether or not, for example, a visitor or the like is wearing a commercially available sanitary mask in an appropriate manner. That is, the determination unit 18 is configured to determine whether or not the sanitary mask worn by the user satisfies a predetermined condition corresponding to the type of user.

Users who visit hospitals include outpatients who visit for medical examinations, outpatients who visit patients, MRs (medical representatives) who visit doctors, carriers who bring in medicines, medical equipment and supplies, etc. There are a wide variety of services, and the needs of each user are different. Therefore, for example, a manager or an operator can store a plurality of pieces of condition information in the storage unit 11 for each user's business. For example, if the purpose of a user visiting a hospital is to see a doctor or visit a patient, it is possible to set the sanitary mask condition information more strictly than for other purposes. In this case, the user inputs the matter to the matter input unit 2, and the condition information acquisition unit 17 reads the condition information from the storage unit 11 according to the matter. The determination unit 18 is configured to determine whether or not the sanitary mask worn by the user satisfies a predetermined condition corresponding to the task.

For example, in places other than hospitals, where written exams such as university entrance exams are conducted, the wearing of sanitary masks with letters is often prohibited from the viewpoint of fraud prevention. In such a case, the condition information is set as "no text", "monochromatic", etc., and the determination unit 18 determines whether the sanitary mask worn by the user is monochromatic. Also, it is configured to determine whether or not the sanitary mask has characters. Note that the determining unit 18 may be configured to determine that the predetermined condition is satisfied only when specific characters are attached to the sanitary mask even when the characters are attached.

For example, in the case of an area where the city is blocked or the flow of people is restricted, the condition information may be set more severely than in an area where the city is not blocked or the flow of people is not restricted. In such a case, the storage unit 11 is configured to store a plurality of pieces of condition information corresponding to regions. At this time, the determination unit 18 may be configured to determine whether or not the user is wearing the sanitary mask more appropriately. That is, the determination unit 18 is configured to determine whether or not the sanitary mask worn by the user satisfies a predetermined condition corresponding to the region.

As a measure to prevent the spread of COVID-19, restaurants have restricted their business mode by time. When dealing with such an event, the storage unit 11 is configured to store a plurality of pieces of condition information corresponding to time periods. At this time, the determination unit 18 may be configured to determine whether or not the user is wearing the sanitary mask more appropriately during a specific time period. That is, the determination unit 18 is configured to determine whether or not the sanitary mask worn by the user satisfies the predetermined condition corresponding to the time period.

Based on the above specific example, an example of the flow of processing for determining suitability of a sanitary mask by the image processing system of the embodiment shown in FIG. 1 will be described based on FIG. First, when the face detection unit 13A detects the face information area A2 in which the user's face exists in the captured image data of the imaging area A1 captured by the video input unit 12 (step #01), the user identification unit 15 identifies the user. Then, the user type acquisition unit 14 acquires user information indicating the type of user (step #02).

Next, the mask detection unit 13B attempts to detect a mask existence area A3 where a sanitary mask exists from the face information area A2 (step #03). Step #03 corresponds to the "detection step" of the present invention. If the user wears a sanitary mask, the mask presence area A3 is detected, and if the user does not wear the sanitary mask, the mask presence area A3 is not detected. If the mask detection unit 13B does not detect the sanitary mask (step #03: No), information indicating that the sanitary mask is not detected is output to the notification unit 19 via the mask information acquisition unit 16 and the determination unit 18. Then, the notification unit 19 controls the notification device 4 to notify the user to wear the sanitary mask (step #04).

When the mask detection unit 13B detects the sanitary mask (step #03: Yes), the mask information acquisition unit 16 acquires information (sanitary mask information) on the sanitary mask shown in the mask existing area A3 (step #05). Step #05 corresponds to the "acquisition step" of the present invention.

A plurality of pieces of condition information are stored in the storage unit 11, and information for the condition information acquisition unit 17 to read appropriate condition information is acquired in steps #06 to #08. The area information acquisition unit 3 acquires area information about the area where the camera 1 is installed (step #06). This area information is sent to the condition information acquisition unit 17 .

Also, the condition information acquisition unit 17 acquires the current time (step #07). The matter input unit 2 acquires information on the matter of the user (step #08). Information about this matter is sent to the condition information acquisition unit 17 .

If the processing of steps #06 to #08 is unnecessary, the processing of steps #06 to #08 may be omitted, and at least one of steps #06 to #08 is performed as necessary. It may be a configuration.

The condition information acquisition unit 17 acquires condition information according to the user's type, region, time (time zone), business matter, etc., from the storage unit 11 (step #09). Then, the determination unit 18 determines whether or not the sanitary mask worn by the user satisfies a predetermined condition (step #10).

If the predetermined condition is satisfied (step #10: Yes), the notification unit 19 notifies the user that the sanitary mask satisfies the predetermined condition via the notification device 4 (step #11). If the predetermined condition is not satisfied (step #10: No), the notification unit 19 notifies the user via the notification device 4 that the sanitary mask does not satisfy the predetermined condition and the reason therefor. (step #12).

When the mask detection unit 13B does not detect the sanitary mask (step #03: No) and when the sanitary mask does not satisfy the predetermined condition (step #10: No), the determination unit 18 stores the number of times the predetermined condition is not satisfied for each user in the storage unit 11 (step #13). Note that the determination unit 18 may be configured so as not to perform the process of step #13 when the user identification unit 15 fails to identify the user.

[Image processing system for personal authentication based on sanitary masks]
In this embodiment, the mask information acquisition unit 16 is configured to be able to acquire a large amount of information about the sanitary mask. Therefore, when personal identification information is attached to the sanitary mask, personal authentication based on the sanitary mask is possible. An image processing system for performing personal authentication based on a sanitary mask will be described below with reference to FIGS. 3 and 4. FIG.

FIG. 3 is a block diagram of an image processing system for performing face authentication. The image processing system of the embodiment shown in FIG. 3 includes a camera 1, a notification device 4, and a gate device 5. The camera 1 and the notification device 4 are as described with reference to FIG. The gate device 5 has an openable gate. The image processing system of the embodiment shown in FIG. 3 opens the gate device 5 and permits the user to enter a predetermined area when the face authentication of the user is successful.

The image processing system of the embodiment shown in FIG. unit”), a determination unit 26, a notification unit 27, a signal output unit 28, and a face registration unit 29 are provided. The functional units (storage unit 21 to face registration unit 29) described above are program modules built in an image processing server capable of real-time processing related to image analysis processing, and are configured to operate in a cloud environment. The video input unit 22 is communicably connected to the camera 1, the notification device 4 is communicatively connected to the notification unit 27, and the gate device 5 is communicably connected to the signal output unit .

The storage unit 21 stores information about users who use the facility in a storage medium (for example, hard disk or flash memory). The information includes the user's face information and personal identification information. The video input section 22 has the same configuration as the video input section 12 described with reference to FIG.

The detection section 23 includes a face detection section 23A and a mask detection section 23B. The face detection unit 13A detects the face information area A2 in which the user's face exists, from the captured image data of the imaging area A1 captured by the video input unit 22. FIG. The image of the face information area A2 becomes face information. The mask detection unit 23B detects a mask existence area A3 where a sanitary mask exists from the face information area A2, and detects a mark area A4 with personal identification information from the mask existence area A3. The mask detection section 23B corresponds to the "detection function" of the present invention. The face information detected by face detection section 23A is transmitted to face registration section 29 and determination section 26 . The face information detected by the face detection section 23A is called "first face information". That is, the face detection unit 23A is configured to be able to detect the first face information regarding the user's face from the captured image.

The face registration section 29 is configured to be able to store face information detected from the image of the face information area A2 in the storage section 21 . Specifically, when new employee information is newly registered, the mode of the image processing system is set to registration mode, and new information on users using the facility is registered together with the new employee's new personal identification information. Registered in the storage unit 21 . Then, the face information is stored in association with the new information. The face information stored in the storage unit 21 is called "second face information". Note that the second face information stored in the storage unit 21 may include a plurality of pieces of face information per person. In this case, at least one piece of face information is face information without a sanitary mask.

Note that the face registration unit 29 may be configured to repeatedly store the face information of the user wearing the mask in the storage unit 21 as the second face information each time personal authentication is successful. If the face registration unit 29 is configured to repeatedly store face information in the storage unit 21, the image processing system machine-learns the face information of the user wearing the mask, thereby allowing the user to wear the mask. It is possible to improve the accuracy of face recognition when there is a person.

The mask information acquisition unit 24 detects the marked area A4 with personal identification information from the mask existing area A3 where the sanitary mask exists, and acquires the sanitary mask information on the sanitary mask based on the marked area A4. In the embodiment shown in FIGS. 3 and 4, the sanitary mask information is personal identification information marked on the sanitary mask. The personal identification information is, for example, a two-dimensional code such as a QR code (registered trademark) or data matrix, a bar code, or a color code. That is, the mask information acquisition unit 24 is configured to acquire personal identification information written on the sanitary mask as the sanitary mask information based on the marked area A4 of the mask existing area A3. The mask information acquisition unit 24 then transmits the personal identification information to the user identification unit 25 and the determination unit 26 . The mask information acquisition unit 24 corresponds to the "acquisition unit" and "acquisition function" of the present invention.

The user identification unit 25 receives the personal identification information from the mask information acquisition unit 24, identifies the user stored in the storage unit 21 based on the personal identification information, and obtains the second face information of the user from the storage unit 21. read out. That is, the user identification unit 25 is configured to identify the second face information regarding the user's face based on the personal identification information. The user identification unit 25 then transmits the second face information read from the storage unit 21 to the determination unit 26 .

The determination unit 26 is provided with a face matching unit 26A and a re-wear confirmation unit 26B. The re-wear confirmation section 26B will be described later.

The face verification unit 26A is configured to perform face authentication by verifying the first face information acquired by the face detection unit 23A and the second face information read from the storage unit 21 by the user identification unit 25. It is Specifically, the face matching unit 26A calculates the degree of similarity between the first face information and the second face information. When the similarity is higher than a preset threshold value, the face matching unit 26A determines whether the authentication is successful (the person "user" associated with the first face information matches the person "registrant" associated with the second face information). ). As a result, the determination unit 26 allows the user to access the predetermined area based on the two-factor authentication of the personal identification information (possession authentication) and the face authentication (biometric authentication) result of the face matching unit 26A. Determine whether or not to permit entry.

Based on FIG. 4, an example of the flow of processing for performing personal authentication by the image processing system of the embodiment shown in FIG. 3 will be described. First, when the face detection section 23A detects the face information area A2 in which the user's face exists in the captured image data of the imaging area A1 captured by the video input section 22 (step #21), the mask detection section 23B attempts to detect a mask existing area A3 where a sanitary mask exists from the face information area A2 (step #22). Step #22 corresponds to the "detection step" of the present invention. If the user wears a sanitary mask, the mask presence area A3 is detected, and if the user does not wear the sanitary mask, the mask presence area A3 is not detected. At this time, the face detection unit 23A detects the first face information from the captured image and transmits the first face information to the determination unit 26. FIG.

When the mask detection unit 23B detects the sanitary mask (step #22: Yes), the mask information acquisition unit 24 attempts to detect the mark area A4 from the sanitary mask projected onto the mask existence area A3 (step #23). If the marked area A4 exists, the mask information obtaining section 24 can obtain personal identification information. Step #23 corresponds to the "acquisition step" of the present invention. If the mask detection unit 23B does not detect the sanitary mask (step #22: No) and if the mask information acquisition unit 24 cannot detect the personal identification information from the sanitary mask (step #23: No), the personal identification information is detected. A signal indicating that it is impossible is output to the notification unit 27 via the determination unit 26 . Then, the notification unit 27 controls the notification device 4 to notify the user to wear a regular sanitary mask (step #24).

When the mask information acquisition unit 24 detects personal identification information from the sanitary mask (step #23: Yes), the user identification unit 25 identifies second facial information based on the personal identification information (step #25). Furthermore, the user identification unit 25 transmits the second face information to the determination unit 26 . In addition, since the face detection section 23A transmits the first face information to the determination section 26, the determination section 26 acquires the first face information (step #26).

When the determination unit 26 acquires the first face information and the second face information, the face collation unit 26A collates the first face information and the second face information (steps #27 and #28). When the user is wearing the sanitary mask, the face matching unit 26A checks a portion of the user's face area above the sanitary mask (for example, the eyes). In other words, the face matching unit 26A calculates the degree of similarity between the first face information and the second face information in the portions above the sanitary mask. Then, the face matching unit 26A determines that authentication is successful when the degree of similarity is higher than a preset threshold.

When the first face information and the second face information match in the face authentication performed by the face matching section 26A (step #28: Yes), the determination section 26 determines that personal authentication has succeeded. Then, the notification unit 27 notifies the user of the success of face authentication via the notification device 4 (step #29). The signal output unit 28 is configured to control the opening and closing of the gate device 5 . When the first face information and the second face information match, the signal output section 28 controls the opening of the gate device 5 (step #30). As a result, the gate device 5 is opened, and the image processing system of this embodiment permits the user to enter the predetermined area.

When the face matching unit 26A matches only the part of the user's face area above the sanitary mask, the accuracy of matching may decrease, and the face matching unit 26A may determine authentication failure. In this case, the face matching unit 26A is configured to perform face authentication using face information obtained when the user is not wearing a sanitary mask.

Specifically, when the face matching unit 26A fails in face authentication (step #28: No), the notification unit 27 notifies the user via the notification device 4 to remove the sanitary mask (step # 31). In other words, the notification unit 27 notifies the user whether or not the face authentication is successful, and also notifies the user to remove the sanitary mask when the face matching unit 26A fails the face authentication.

While the notification unit 27 is performing notification, the mask detection unit 23B attempts to detect the mask existence area A3 where the sanitary mask exists from the face information area A2 (step #32). The detection result of the mask detection section 23B is transmitted to the determination section 26, and the face matching section 26A of the determination section 26 determines whether or not the user is removing the sanitary mask. When the user wears a sanitary mask, the mask presence area A3 is detected by the mask detector 23B. A determination of Yes is made in step #31, and the processes of steps #31 and #32 are repeated until the mask existence area A3 is no longer detected.

When the mask detection unit 23B no longer detects the sanitary mask (step #32: No), it becomes possible to acquire a captured image of the user without the mask. Then, the face detection unit 23A detects the user's face information from the captured image in which the user removes the sanitary mask. The face information when the user removes the sanitary mask is called "third face information". The face detection section 23A then transmits the detected third face information to the determination section 26 . That is, the determination unit 26 acquires the third face information from the captured image (step #33).

When the determination unit 26 acquires the third face information, the face collation unit 26A collates the second face information and the third face information (step #34, step #35). Specifically, the face matching unit 26A compares the second face information and the third face information to calculate the degree of similarity. Since the third facial information is acquired with the sanitary mask removed, this collation is a collation using the entire face. Then, the face matching unit 26A determines that authentication is successful when the degree of similarity is higher than a preset threshold. That is, the face matching unit 26A is configured to perform face authentication by matching the second face information and the third face information when the face authentication by matching the first face information and the second face information fails. It is With this configuration, the face matching section 26A can perform highly accurate face authentication. The matching result of the face matching unit 26A is transmitted from the determination unit 26 to the notification unit 27. FIG.

When the face matching unit 26A determines that the second face information and the third face information do not match (step #35: No), the notification unit 27 controls the notification device 4 to notify the user of personal authentication failure ( step #36). At this time, if the gate device 5 is open to allow entry, the signal output unit 28 may be configured to control the gate device 5 to be closed.

When the face matching unit 26A determines that the second face information and the third face information match (step #35: Yes), the notification unit 27 controls the notification device 4 to notify the user of the success of face authentication ( step #37). At this time, it is conceivable that the user has taken off the sanitary mask. Therefore, in the embodiment shown in FIGS. 3 and 4, the determination section 26 is provided with a re-wear confirmation section 26B.

After the face recognition unit 26A has successfully performed face recognition by matching the second face information and the third face information, the notification unit 27 controls the notification device 4 to notify the user to wear a sanitary mask (step #38). ).

While the notification unit 27 is notifying, the mask detection unit 23B attempts to detect the mask existence area A3 where the sanitary mask exists from the face information area A2, and the re-wearing confirmation unit 26B confirms whether the user is re-wearing the sanitary mask. (step #39). The detection result of the mask detection unit 23B is transmitted to the determination unit 26, and the re-wear confirmation unit 26B of the determination unit 26 determines whether or not the user is wearing a sanitary mask after the face authentication is successful. If the user is wearing a sanitary mask, the mask presence area A3 is detected by the mask detection unit 23B. And the process of step #39 is repeated.

When the re-wear confirmation unit 26B determines that the user is wearing the sanitary mask again (step #39: Yes), the signal output unit 28 controls the opening of the gate device 5 (step #30). As a result, the gate device 5 is opened, and the image processing system of this embodiment permits the user to enter the predetermined area.

[Reference example: room entry determination based on changes in the number of pixels between eyes]
Based on FIG.5 and FIG.6, the room entry determination based on the captured image imaged by the camera 1 is demonstrated. In the reference examples shown in FIGS. 5 and 6, an entrance E of a facility, a point D located outside the facility with respect to the entrance E, and a point F located inside the facility with respect to the entrance E are shown. there is The point D, the entrance E and the point F are located on a straight line (or almost on a straight line) along the traveling route of the user, and the camera 1 is installed on the extension of the straight line in the vicinity of the point F. . The camera 1 is configured to image the front of the user passing through the entrance E. As shown in FIG.

Specifically, before the user enters the entrance E of the facility, the user is imaged by the camera 1, and the user's movement in the depth direction is captured in the captured image. A pair of eyes of the user is detected from the captured image, and the number of inter-eye pixels indicating the distance between the pair of eyes is calculated. As the user approaches the entrance E of the facility, the distance between the user and the camera 1 becomes smaller, the user appears larger in the captured image, and the number of inter-eye pixels increases. Whether the user enters the room is determined based on the number of pixels between the eyes, and when the number of pixels between the eyes increases to a predetermined threshold value or more, it is determined that the user has entered the room.

The function of calculating the number of inter-eye pixels and the function of determining whether the user has entered the room may be implemented in a computer built into the camera 1, may be implemented in a computer other than the camera 1, or may be implemented in the cloud. It may be configured to operate in the environment.

In the reference example shown in FIG. 5, a fixed value is adopted as the threshold value for room entry determination, and the predetermined fixed value is set to, for example, 40 pixels. Then, when the number of inter-eye pixels reaches the predetermined fixed value or more, it is determined that the user has passed through the entrance E and entered the room. However, there are individual differences in the distance between a pair of eyes, and if the number of pixels between the eyes is used for room entry determination, the error in room entry determination increases due to the individual differences. For example, a person with a large physique and a large distance between their eyes is determined to have entered the room before the entrance E (between the point D and the entrance E). A decision error L1 is shown in FIG. 5 as an error in room entry decision.

The reference example shown in FIG. 6 is an improved measure of the reference example shown in FIG. In this example, the rate of increase in the number of inter-eye pixels is used for room entry determination. Specifically, the rate of increase in the number of pixels between eyes is calculated based on the number of pixels between eyes when the user's feet straddle point D as a reference. An increase rate of 1.5 is set as the threshold for room entry determination. As the user approaches the entrance E of the facility, the number of inter-eye pixels increases, and the rate of increase in the inter-eye pixel number also increases. Then, when the rate of increase in the number of inter-eye pixels exceeds the threshold value (1.5), it is determined that the user has entered the room. A decision error L2 is shown in FIG. 6 as an error in room entry decision. The determination error L2 is smaller than the determination error L1. In the reference example shown in FIG. 6, the entrance of the user at the entrance E is determined more accurately than in the reference example shown in FIG.

[Another embodiment]
The present invention is not limited to the configurations exemplified in the above embodiments, and other representative embodiments of the present invention will be exemplified below.

(1) In the above embodiment, the camera 1, the subject input unit 2, the area information acquisition unit 3, and the notification device 4 are incorporated into one device, but the configuration is not limited to this embodiment. . For example, the camera 1, the message input unit 2, the local information acquisition unit 3, and the notification device 4 are configured as separate devices, and the camera 1, the message input unit 2, the local information acquisition unit 3, and the notification device 4 are combined. A configuration in which they are interlocked with each other is also possible. In this case, the matter input unit 2 may be, for example, a counter navigation system or an automatic reception machine installed at counters of financial institutions, post offices, hospitals, and the like. The camera 1 may be installed near the automatic reception machine or the like that is the matter input unit 2, and may be configured to capture the face of the user who operates the touch panel monitor of the automatic reception machine or the like.

(2) In the image processing system of the embodiment shown in FIG. 1, the functional units (storage unit 11 to notification unit 19) are built in an image processing server capable of real-time processing and configured to operate in a cloud environment. , is not limited to this embodiment. All the configurations shown in FIG. 1 may be incorporated into one device, such as a tablet computer, and configured as an image processing device.

Similarly, in the image processing system of the embodiment shown in FIG. 3, the functional units (storage unit 21 to face registration unit 29) are built in an image processing server capable of real-time processing, and are configured to operate in a cloud environment. is not limited to this embodiment. All the configurations shown in FIG. 3 may be incorporated into one device, such as a tablet computer, and configured as an image processing device.

In addition, the configurations of both FIG. 1 and FIG. 3 may be incorporated into one device and configured as an image processing device.

(3) In the image processing system of the embodiment shown in FIG. 1, the configuration may be such that the message input unit 2 is not provided. The image processing system of the embodiment shown in FIG. 1 may have a configuration in which the regional information acquisition unit 3 is not provided. That is, the condition information acquisition unit 17 obtains the type of user, the matter input by the matter input unit 2, the area information obtained by the area information acquisition unit 3, and the mask information acquisition unit 16 based on the sanitary mask information. It is sufficient that the condition information is read out from the storage unit 11 based on at least one of the time zone at the acquired timing and the condition information.

(4) In the embodiment shown in FIGS. 1 and 2, the user type acquisition unit 14 acquires the user type based on the user information, but is not limited to this embodiment. For example, the user type acquisition unit 14 may be configured to acquire the type of user based on the user's clothing imaged in the captured image. For example, the user type acquisition unit 14 may be configured to acquire the user type from the user's ID card or the like.

(5) In the embodiment shown in FIGS. 1 and 2, the user identification unit 15 is provided, and the determination unit 18 is configured to store the number of times the sanitary mask did not satisfy the predetermined condition in the storage unit 11 for each user. , but is not limited to this embodiment. The embodiment shown in FIGS. 1 and 2 may have a configuration in which the user identification unit 15 is not provided. In this case, the determination unit 18 may be configured to simply store the date and time when the sanitary mask did not satisfy the predetermined condition in the storage unit 11 in the process of step #13 in FIG. If the user identification unit 15 is not provided, the user information indicating the type of user is acquired at the timing after step #05 and before step #09, not at the timing of step #02 in FIG. It may be a configuration.

(6) The configuration may be such that the re-wearing confirmation section 26B shown in FIG. 3 is not provided. In this case, the processing of steps #38 and #39 in FIG. 4 may be omitted, and the processing of step #30 may be executed immediately after the processing of step #37.

(7) The determination unit 26 described with reference to FIGS. 3 and 4 performs two-factor authentication based on personal identification information (property authentication) and the result of face authentication (biometric authentication) of the face matching unit 26A. , determines whether or not the user is permitted to enter the predetermined area, but is not limited to this embodiment. In the image processing system of the embodiment shown in FIGS. 3 and 4, the determination unit 26 permits the user to enter the predetermined area based only on the result of face authentication (biometric authentication) by the face matching unit 26A. A configuration for determining whether or not is also possible.

The face matching unit 26A described with reference to FIGS. 3 and 4 compares the first face information and the second face information to calculate the similarity, and when the similarity is higher than a preset threshold, Judge as authentication success. If the determining unit 26 is configured to determine whether or not to permit the user to enter the predetermined area based only on the result of face authentication (biometric authentication) by the face matching unit 26A, the similarity threshold is A configuration in which the threshold is set high and this threshold is set as the first threshold may be used. Then, when the determination unit 26 determines whether or not to allow the user to enter the predetermined area based on the two-factor authentication, the similarity threshold is a second threshold lower than the first threshold may be set to .

(8) The embodiments of FIGS. 1 and 3 may be separate image processing systems, or may be integrated as one image processing system. For example, the determination unit 18 shown in FIG. 1 is provided with a face matching unit 26A and a re-wearing confirmation unit 26B shown in FIG. 3, and in the embodiment shown in FIG. It may be a configuration that is In this case, the determination unit 18 shown in FIG. 1 may be configured to be able to perform both determination of suitability of the sanitary mask worn by the user and personal authentication based on face information. In addition, the mask information acquisition unit 16 in this case not only acquires the sanitary mask information based on the mask existing area A3 detected by the mask detection unit 13B, but also acquires sanitary mask information based on the mark area A4 shown in FIG. The configuration may be such that sanitary mask information is acquired and transmitted to the user identification unit 15 . Then, similarly to the user identification unit 25, the user identification unit 15 identifies the user stored in the storage unit 11 based on the personal identification information received from the mask information acquisition unit 16, The configuration may be such that the second face information of the person is read and transmitted to the determination unit 18 .

(9) The message input unit 2 is, for example, a touch panel, and the user who operates the message input unit 2 may sneeze or cough. The image processing system may be configured to detect the user's sneezing or coughing and take action. For example, the camera 1 can detect user's actions such as sneezing and coughing by using a skeleton detection technique or the like. When the notification units 19 and 27 detect sneezing or coughing by a user who is not wearing a sanitary mask, the administrator or staff is notified via the notification device 4 to disinfect the matter input unit 2 (touch panel monitor). It may be configured to instruct.

It should be noted that the configurations disclosed in the above embodiments (including other embodiments, the same shall apply hereinafter) can be applied in combination with configurations disclosed in other embodiments as long as there is no contradiction. Moreover, the embodiments disclosed in this specification are merely examples, and the embodiments of the present invention are not limited thereto, and can be modified as appropriate without departing from the scope of the present invention.

INDUSTRIAL APPLICABILITY The present invention is applicable to an image processing system that processes a captured image of the face of a user wearing a sanitary mask. Moreover, the technical features of the image processing system of the present invention can also be applied to an image processing apparatus. For this reason, the above embodiments can be configured as an image processing apparatus integrated into one apparatus. Furthermore, the technical features of the image processing system of the present invention can also be applied to the image processing method. Therefore, the above embodiment can be configured as an image processing method. Additionally, the technical features of the image processing system of the present invention can also be applied to image processing programs. Therefore, the above embodiment can be configured as an image processing program.

2: Matter input section (input section)
3: area information acquisition unit 4: notification device 11, 21: storage unit 13, 23: detection unit 14: user type acquisition unit 15: user identification unit (first user identification unit)
16, 24: mask information acquisition unit (acquisition unit)
18: Determination unit 19, 27: Notification unit 25: User identification unit (second user identification unit)
26A: face matching unit 26B: re-wear confirmation unit A3: mask existing area

Claims (19)

An image processing system that processes a captured image of the face of a user wearing a sanitary mask,
a detection unit that detects a mask existence area in which the sanitary mask is projected from the captured image;
an acquisition unit that acquires sanitary mask information about the sanitary mask based on the mask existing area. a storage unit that stores condition information indicating a predetermined condition regarding the sanitary mask;
2. The image processing according to claim 1, further comprising a determination unit that compares the sanitary mask information and the condition information and determines whether the sanitary mask worn by the user satisfies the predetermined condition. system. further comprising a user type acquisition unit that acquires user information indicating the type of user,
The storage unit is configured to store a plurality of the condition information corresponding to the type of the user,
The image processing system according to claim 2, wherein the determination unit is configured to determine whether the sanitary mask worn by the user satisfies the predetermined condition corresponding to the type of the user. Further comprising a regional information acquisition unit that acquires the region where the captured image was captured,
The storage unit is configured to store a plurality of the condition information corresponding to the region,
The image processing system according to claim 2 or 3, wherein the determination unit is configured to determine whether the sanitary mask worn by the user satisfies the predetermined condition corresponding to the region. The storage unit is configured to store a plurality of pieces of condition information corresponding to time zones,
5. The determination unit according to any one of claims 2 to 4, wherein the determination unit is configured to determine whether the sanitary mask worn by the user satisfies the predetermined condition corresponding to the time period. image processing system. Further comprising an input unit for the user to input a matter,
The storage unit is configured to store a plurality of the condition information corresponding to the business,
6. The determination unit according to any one of claims 2 to 5, wherein the determination unit is configured to determine whether the sanitary mask worn by the user satisfies the predetermined condition corresponding to the requirement. image processing system. Further comprising a notification unit that controls a notification device to notify the user,
The notification unit notifies the user wearing the sanitary mask whether or not the predetermined condition is satisfied, and when the predetermined condition is not satisfied, notifies the user of the reason why the predetermined condition is not satisfied. 7. An image processing system according to any one of claims 2 to 6, configured to: The detection unit is configured to be able to detect first face information related to the user's face from the captured image,
further comprising a first user identification unit that identifies the user from the first face information;
8. The image processing system according to any one of claims 2 to 7, wherein the determination unit is configured to be able to store the number of times the predetermined condition is not satisfied for each user in the storage unit. 2. The acquisition unit is configured to acquire at least one of a color and a shape of the sanitary mask worn by the user as the sanitary mask information based on the mask existing area. 9. The image processing system according to any one of 8. 10. The acquisition unit is configured to acquire at least one of a position and an angle of the sanitary mask on the user's face as the sanitary mask information based on the mask existing area. The image processing system according to any one of . The image processing system according to any one of claims 1 to 10, wherein the acquisition unit is configured to acquire the type of the sanitary mask as the sanitary mask information based on the mask existence area. 12. The acquisition unit according to any one of claims 1 to 11, wherein the acquisition unit is configured to acquire personal identification information written on the sanitary mask as the sanitary mask information based on the mask existence area. image processing system. The detection unit is configured to be able to detect first face information related to the user's face from the captured image,
a second user identification unit that identifies second face information related to the user's face based on the personal identification information;
13. The image processing system according to claim 12, further comprising a face matching unit that performs face authentication of the user by matching the first face information and the second face information. Further comprising a notification unit that controls a notification device to notify the user,
The notification unit notifies the user whether or not the face authentication is successful, and notifies the user to remove the sanitary mask when the face matching unit fails the face authentication. death,
The detection unit is configured to be able to detect third face information related to the user's face from the captured image when the user has removed the sanitary mask,
14. The image processing system according to claim 13, wherein the face matching section is configured to perform the face authentication by matching the second face information and the third face information. 15. The image processing system according to claim 14, wherein when the face authentication is successful, the notification unit notifies the user to wear the sanitary mask. 16. The image processing system according to claim 15, further comprising a re-wear confirmation unit that determines whether or not the user is wearing the sanitary mask after the face authentication is successful. An image processing method for processing a captured image of the face of a user wearing a sanitary mask,
a detection step of causing a computer to detect a mask existence area where the sanitary mask is projected from the captured image;
an obtaining step of obtaining, in a computer, sanitary mask information regarding the sanitary mask based on the mask existing area. An image processing program that causes a computer to process a captured image of the face of a user wearing a sanitary mask,
A detection function that detects a mask existence area where the sanitary mask is projected from the captured image;
An image processing program that causes a computer to execute an acquisition function of acquiring sanitary mask information related to the sanitary mask based on the mask existing area. An image processing device that processes a captured image of the face of a user wearing a sanitary mask,
a detection unit that detects a mask existence area in which the sanitary mask is projected from the captured image;
an acquisition unit configured to acquire sanitary mask information about the sanitary mask based on the mask existence area.

Images