JP2024030353A - Determination device, determination method, and program - Google Patents

Abstract

To provide a determination device capable of properly determining whether a captured image of the outside of a predetermined area is genuine or not.SOLUTION: A determination device 10 comprises: a first video acquisition unit (a first image acquisition unit) 11 which acquires a first video (a first image) captured by an image capturing unit 3 for capturing an image of the outside of a predetermined area; a second video acquisition unit (a second image acquisition unit) 12 which acquires a second video (a second image) captured by the image capturing unit 3 before capturing the first video; and a determination unit 15 which compares the first video with the second video and in the case of a predetermined variation or more, determines that the first video is a false video (a false image) which is not a video actually captured outside.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to a determination device, a determination method, and a program.

2. Description of the Related Art Cameras are known that can be installed on front door scopes, intercoms, etc. to take pictures of visitors. By using such a camera, a resident can open the front door after checking the photographed image inside the residence and grasping the appearance of the visitor.

As a related technique, for example, Patent Document 1 discloses a control system that can determine whether a subject of a face image extracted from a photographed image is a real face. For example, when the subject is a face in a photograph or a head covering such as a mask or mask, the control system determines that the subject of the facial image is not a real face and determines that the subject is an impersonation. The system determines that the visitor is an impostor, for example, when the positions of a plurality of feature points in the facial image do not change for a certain period of time.

JP2022-063783A

Suppose that in a system such as that disclosed in Patent Document 1, a malicious person disguises the actual external situation by holding up a picture or video to a door scope or intercom camera when calling a resident. In such cases, there is a risk that a person with malicious intent may cause disadvantage to residents. For example, a situation may arise in which a person who wants to make a hard sale is trying to trick a police officer into opening a door by holding the camera up to an image of the police officer. Furthermore, for example, a situation can be considered in which a person intending to assault or rob a resident holds up an image of another person to force the door to open and attack a resident.

As described above, related technologies have a problem in that if the camera cannot accurately capture the actual external situation, the resident may not be able to correctly recognize the visitor and may be disadvantaged.

In view of the above-mentioned problems, an object of the present disclosure is to provide a determination device, a determination method, and a program that can appropriately determine whether an image taken outside a predetermined area is genuine. .

The determination device according to the present disclosure includes:
a first image acquisition section that acquires a first image photographed by a photographing section that photographs the outside of a predetermined area;
a second image acquisition unit that acquires a second image photographed by the photographing unit before photographing the first image;
a determination unit that compares the first image and the second image, and determines that the first image is a false image that is not an image actually photographed outside if there is a change greater than a predetermined value; It is equipped with the following.

The determination method according to this disclosure is:
The computer is
a first image acquisition step of acquiring a first image photographed by a photographing unit photographing the outside of the predetermined area;
a second image acquisition step of acquiring a second image photographed by the photographing unit before photographing the first image;
a determination step of comparing the first image and the second image, and determining that the first image is a false image that is not an image actually photographed outside if there is a change greater than a predetermined value; It is intended to carry out the following.

The program related to this disclosure is
a first image acquisition step of acquiring a first image photographed by a photographing unit photographing the outside of the predetermined area;
a second image acquisition step of acquiring a second image photographed by the photographing unit before photographing the first image;
a determination step of comparing the first image and the second image, and determining that the first image is a false image that is not an image actually photographed outside if there is a change greater than a predetermined value; This is what causes a computer to execute.

A determination device, a determination method, and a program according to the present disclosure make it possible to appropriately determine whether an image taken outside a predetermined area is genuine.

FIG. 1 is a block diagram showing the configuration of a determination system according to an embodiment. 2 is a flowchart showing the overall flow of processing performed by the determination device according to the embodiment. 3 is a flowchart showing a first determination method according to the embodiment. It is a flow chart which shows the 2nd judgment method concerning an embodiment.

Embodiments of the present disclosure will be described in detail below with reference to the drawings. Identical or corresponding elements are provided with the same reference numerals in the drawings. For clarity of explanation, redundant explanations will be omitted as necessary.

(Configuration of determination system 200)
First, with reference to FIG. 1, a determination system 200 according to the present embodiment will be described. FIG. 1 is a block diagram showing the configuration of a determination system 200 according to this embodiment. As shown in FIG. 1, the determination system 200 includes a door scope 1, an intercom button 2, a photographing section 3, a video recording section 4, an illuminance sensor 5, an environmental information DB 6, a locking control section 7, a notification section 8, and a determination device. It is equipped with 10.

In the determination system 200, the imaging unit 3 photographs the outside of a predetermined area, and the determination device 10 uses the plurality of acquired images to determine whether the image taken outside the predetermined area is genuine. It is an information processing system that Further, in this embodiment, the photographing unit 3 photographs a video (moving image). The determination device 10 compares a plurality of videos taken at different timings and determines whether the video taken by the imaging unit 3 is a fake video.

In the following, a real image in which the outside is actually photographed by the photographing unit 3 may be referred to as a "true image." Furthermore, hereinafter, an image that is not a true image may be referred to as a "fake image". A fake video is, for example, a disguised video that shows a person other than the person who actually visited the site. The fake video may be, for example, a video that includes a person pretending to be a package delivery person. If such a false image is captured by the imaging unit 3, there is a possibility that the resident may open the door erroneously recognizing that the image is a real image.

The false video may have been previously photographed at a position different from the photographing position of the photographing unit 3. In such cases, the fake video may include objects or roads that do not actually exist in the surrounding area. Additionally, the fake video may include an image showing weather that is different from the actual outside. For example, the actual weather may be rainy, but the fake video may show that the sky is clear.

The present disclosure allows the determination device 10 to perform determination processing using such differences between images, thereby making it possible to appropriately determine whether an image taken outside a predetermined area is genuine. do.

The predetermined area is a space where a user who receives a call from a visitor exists. The predetermined area is, for example, a residence. The predetermined area may be a condominium, an apartment, a store, an accommodation facility, a public facility, or the like. The predetermined area is not limited to these, and may be any of various spaces that can receive calls from visitors. A user who receives a call from a visitor may be, for example, a resident of a predetermined area, an administrator, or a user, but is not limited to these.

Moreover, the outside of the predetermined area indicates the area outside the boundary part that constitutes the predetermined area. The boundary portion is, for example, a wall of a residence. An entrance/exit for entering/exiting the predetermined area is provided at the boundary. The entrance/exit is, for example, an entrance door provided at the entrance. The outside of the predetermined area is, for example, an area near the entrance door. The outside of the predetermined area may be, for example, a lot in front of the entrance door, a passage, or the like. Further, for example, in a housing complex, the outside of the predetermined area may be an entrance provided in front of a common entrance door. The outside of the predetermined area may be outdoors or indoors.

Note that in the following description, the outside of the predetermined area may be simply referred to as "outside". Further, the entrance door may be simply referred to as a "door". Note that the entrance/exit may be provided in a form other than a door.

Here, as an example, an example in which the determination system 200 is used in a residence will be described. The outside of the predetermined area is the area outside the front door provided in the residence. Further, the user is a resident of the residence.

The door scope 1 is provided at the entrance door. The door scope 1 is configured so that a resident inside the residence can check the visitor outside the door without opening the front door. The door scope 1 may be configured using, for example, an optical system having a scope lens including an objective lens facing the outside of the door, an eyepiece lens facing the inside of the door, and the like. By looking directly into the door scope 1, the resident can check the visitor outside the door and the surroundings.

The interphone button 2 is a button operated by a visitor to call a resident. The interphone button 2 is provided, for example, on a wall near the door. The determination system 200 is configured so that when a visitor presses the intercom button 2, a ring tone or the like sounds inside the residence. Residents recognize that a visitor has arrived by hearing the ringtone.

The photographing unit 3 is a photographing device that photographs the outside of a predetermined area. The photographing section 3 acquires a photographed image and outputs it to the video recording section 4. The photographing unit 3 is, for example, a camera. The photographing unit 3 may be, for example, an infrared camera capable of photographing even in a dark place. The photographed image may be a video (moving image) or a still image. In this embodiment, the description will be given assuming that the photographed image is a video. Note that in the present disclosure, "video" may mean video data that is data of the video.

The photographing unit 3 is provided in the door scope 1, for example. The photographing unit 3 photographs the outside from the position of the door scope 1. Further, the photographing unit 3 photographs the visitor who looks into the door scope 1.

The present invention is not limited to this, and the imaging unit 3 may be provided on a wall near the door. In this case, the photographing section 3 photographs the outside in a predetermined direction from the installed position. Further, when a visitor visits the residence, the photographing unit 3 photographs the visitor. Further, a plurality of photographing units 3 may be provided. For example, by providing the photographing section 3 in the door scope 1 and in the vicinity of the door, it is possible to photograph the visitor from different angles.

The photographing unit 3 photographs the outside at predetermined time intervals. In this embodiment, the photographing unit 3 constantly photographs the outside. Further, the photographing section 3 outputs the acquired video to the video recording section 4.

The video recording section 4 records the video received from the photographing section 3. The video recording unit 4 can temporarily hold recorded video. For example, the video recording unit 4 deletes the recorded video when a predetermined condition is met. The predetermined condition may be, for example, that a predetermined period of time has passed since the video was captured.

If the determination unit 15 (described later) determines that the video is a fake video, the video recording unit 4 does not delete the video for a predetermined period of time (for example, 5 minutes) before the start of shooting the fake video, and Hold. Thereby, when a fake video is shot, the video recording unit 4 can record and hold the video from the timing before the fake video was shot. The video recording unit 4 may hold video for a predetermined period of time after the shooting of the fake video is started. The video recording unit 4 may hold the video up to the timing when the shooting of the fake video ends. The start timing and end timing of the recorded video may be changed as appropriate.

The illuminance sensor 5 is a sensor that acquires illuminance information indicating the actual brightness of the outside. The illuminance sensor 5 may be provided near the door, for example. The illuminance sensor 5 detects ambient illuminance. The illuminance sensor 5 acquires illuminance information b11 as a detection result, and outputs it to the external environment estimation section 14.

As the illuminance sensor 5, various sensors capable of detecting ambient brightness may be used. For example, an infrared sensor, an ultraviolet sensor, or the like may be used as the illuminance sensor 5. Further, instead of the illuminance sensor 5, a solar cell may be used. The ambient illuminance can be detected using the output voltage of the solar cell as an illuminance detection signal. The output voltage changes depending on the radiation intensity of the light receiving surface. This allows the solar cell to detect illuminance depending on the weather.

The illuminance sensor 5 can acquire illuminance information b11 at predetermined time intervals. Further, the illuminance sensor 5 may acquire the illuminance information b11 using the pressing of the interphone button 2 as a trigger. Thereby, the illuminance information b11 can be acquired in real time. Alternatively, the illuminance sensor 5 may detect that a visitor approaches the door using a human sensor (not shown), and acquire the illuminance information b11. Further, the illuminance sensor 5 may detect that the visitor looks into the door scope 1 and acquire the illuminance information b11.

The environment information DB6 stores environment information b12. The environment information b12 may include external weather information B10b indicating the weather around the residence. The external weather information B10b is, for example, information such as fair weather, temperature, humidity, or wind speed around the residence. The environmental information DB 6 acquires external weather information B 10b from a website that provides weather information, for example, via the Internet.

Furthermore, the environmental information DB 6 can store information in which the external weather information B 10b is associated with information regarding ambient brightness as a predetermined table. The environmental information DB 6 may acquire information regarding the surrounding brightness together with the external weather information B 10b from a website or the like. Alternatively, the environmental information DB 6 may obtain only the external weather information B 10b from the website, and obtain the brightness corresponding to the weather information by referring to another table or the like.

The environment information DB 6 is not limited to this, and may acquire other information indicating a correlation between the external weather information B 10b and the surrounding brightness. For example, the environmental information DB 6 may acquire the surrounding brightness using a trained model generated using AI (Artificial Intelligence) that introduces a neural network. For example, the trained model may be trained to input external weather information B10b and output ambient brightness.

The environment information DB 6 outputs the environment information b12 to the external environment estimation unit 14 in response to a request from the external environment estimation unit 14. Further, in this embodiment, the determination system 200 includes both the illuminance sensor 5 and the environmental information DB 6, but it may include either one of them.

The lock control unit 7 controls the locking of doors, windows, other entrances, etc. provided in the residence. The lock control unit 7 includes a communication unit (not shown), and the communication unit connects the determination system 200 to a communication network (not shown). The communication network may be wired or wireless.

The communication unit may be a communication interface for communicating using a wireless standard such as Bluetooth (registered trademark) or Wi-Fi (registered trademark). Further, the communication unit may be a public telephone line network, a dedicated telephone line network, a mobile phone line network, or the like. Since these are just examples, the communication unit can communicate using various communication methods. Moreover, thereby, the communication unit enables communication between the determination system 200 and an external system other than the determination system 200.

The external system may be, for example, a security system installed in a residence. The locking control unit 7 uses the security system to lock doors, windows, etc. when a fake video is captured. Further, the locking control unit 7 may perform control to close open doors and windows. Further, the locking control unit 7 may perform control to unlock doors, windows, etc. by determining whether or not there is no longer a risk of disadvantage to the resident.

The notification unit 8 provides notification when it is determined that a fake video has been shot. The notification unit 8 is, for example, a speaker capable of outputting a warning sound. When it is determined that a false image has been captured, the notification unit 8 sounds an alarm from near the door. This makes it possible to warn visitors. Additionally, the notification unit 8 may output a warning message such as "A disguised video has been taken." in the form of audio. Note that the form of notification is not limited to audio output. The notification unit 8 may display the above warning message on the display.

Further, the notification unit 8 may be configured to report to the police, a security company, etc. via a communication network. Further, the notification unit 8 may notify information terminals used by residents. The information terminal may be, for example, a mobile phone terminal, a smartphone, a tablet terminal, or a PC (Personal Computer).

(Configuration of determination device 10)
Continuing, the configuration of the determination device 10 will be explained with reference to FIG. The determination device 10 includes a first video acquisition section (first image acquisition section) 11, a second video acquisition section (second image acquisition section) 12, a video environment estimation section (first estimation section) 13, and an external environment estimation section ( It includes a second estimation section) 14, a determination section 15, and a registrant DB 16.

The determination device 10 includes a processor, a memory, and a storage device (not shown). The storage device stores a computer program in which the processing according to the present embodiment is implemented. The registrant DB 16 may be stored in the storage device.

The processor can load a computer program into memory from the storage device and execute the computer program. Thereby, the processor realizes the functions of the first video acquisition section 11, the second video acquisition section 12, the video environment estimation section 13, the external environment estimation section 14, and the determination section 15.

Alternatively, the first video acquisition unit 11 and the like may each be realized by dedicated hardware. Further, a part or all of each component of each device may be realized by a general-purpose or dedicated circuit, a processor, etc., or a combination thereof. These may be configured by a single chip or multiple chips connected via a bus. A part or all of each component of each device may be realized by a combination of the circuits and the like described above and a program. Further, as the processor, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (field-programmable gate array), a quantum processor (quantum computer control chip), etc. can be used.

In addition, when a part or all of each component of the determination device 10 is realized by a plurality of information processing devices, circuits, etc., the plurality of information processing devices, circuits, etc. may be centrally arranged or distributed. may be placed. For example, information processing devices, circuits, etc. may be realized as a client server system, a cloud computing system, or the like, in which each is connected via a communication network. Further, the functions of the determination device 10 may be provided in a SaaS (Software as a Service) format.

The first video acquisition section 11 acquires a first video (first image) A1 photographed by the photographing section 3 that photographs the outside of a predetermined area. For example, the first video acquisition section 11 acquires the first video A1 photographed by the photographing section 3 when the interphone button 2 is pressed. The first video A1 includes a video of the visitor who pressed the interphone button 2. Further, when the visitor holds up a fake image to the photographing unit 3, the first image A1 includes the fake image. Note that in this embodiment, when the first video A1 includes a fake video, it is expressed as "the first video A1 is a fake video." The fake video may include an image of a person.

The first video acquisition unit 11 analyzes the first video A1 and outputs the analysis result to the determination unit 15. The first image acquisition unit 11 extracts the person included in the first image A1 using a well-known image recognition technique or the like. The first video acquisition unit 11 outputs the extracted person information to the determination unit 15. The information is, for example, facial image information of the extracted person. The face image information may include information on feature points of the face.

The second video acquisition section 12 acquires a second video (second image) B1 photographed by the photographing section 3 before photographing the first image. The second video B1 is stored in the video recording section 4. The second video acquisition unit 12 acquires the second video B1 in a predetermined period from the video recording unit 4.

Here, the timing at which the interphone button 2 is pressed is indicated by time T1. The second video acquisition unit 12 acquires images taken before time T1 from the video recording unit 4. For example, the second video acquisition unit 12 acquires the second video B1 that was photographed a predetermined time before time T1. For example, the second video acquisition unit 12 acquires the second video B1 for a period from 10 minutes before time T1 to 5 minutes before time T1.

Alternatively, the second video acquisition unit 12 may analyze the second video B1, find the timing at which the visitor appears, and acquire the second video B1 before the timing.

The video environment estimating unit 13 acquires the first video A1 from the imaging unit 3, and estimates a video environment indicating the environment in which the first video A1 was shot based on the first video A1. The video environment estimation unit 13 generates video environment information A10 as the estimation result and outputs it to the determination unit 15.

The video environment information A10 may include video brightness information A10a (first brightness information) indicating the external brightness corresponding to the first video A1. The image brightness information A10a corresponds to the brightness of the first image A1. The video environment estimation unit 13 estimates video brightness information A10a based on the first video A1.

Specifically, first, the video environment estimating unit 13 acquires the first video A1 from the photographing unit 3 when the interphone button 2 is pressed. The video environment estimation unit 13 analyzes the first video A1 and detects a person included in the video scene A1a of the first video A1. The video scene A1a shows an image at a certain time in the first video A1.

Here, the area including the person in the video scene A1a is defined as a person area. The video environment estimating unit 13 specifies an area other than the person area in the video scene A1a as a determination area.

For example, the video environment estimation unit 13 detects a sky region included in the video scene A1a, and specifies the sky region as a determination region. Specifically, the video environment estimation unit 13 generates a histogram showing the frequency distribution of pixels with respect to color values (for example, RGB values) of the video scene A1a. The video environment estimation unit 13 extracts a sky area based on the histogram and specifies the area as a determination area.

The video environment estimation unit 13 obtains video brightness information A10a indicating the external brightness corresponding to the video scene A1a from the color values of pixels included in the determination area. For example, when the weather shown in the video scene A1a is sunny, the brightness shown by the video brightness information A10a is greater than when the weather is rainy or cloudy. Further, for example, when the time slot shown in the video scene A1a is daytime, the brightness indicated by the video brightness information A10a is higher than when the time slot is night time.

The video environment estimating unit 13 may obtain a plurality of video scenes A1a from the first video A1, and obtain video brightness information A10a using the plurality of video scenes A1a.

Note that the video environment estimating unit 13 may detect other subjects and specify the determination area. Further, the video environment estimating unit 13 may obtain the video brightness information A10a using a method other than the above.

Further, the video environment information A10 may include video weather information A10b indicating the external weather in the first video A1. For example, the video environment estimation unit 13 estimates the external weather in the first video A1 based on the image of the determination area. The video weather information A10b may include information such as sunny, cloudy, rain, thunder, snow, or strong wind, for example. The video environment estimation unit 13 may estimate the video weather information A10b by detecting that the clothes of the person appearing in the first video A1 are wet with rain.

The video environment information A10 may include either the video brightness information A10a or the video weather information A10b, or both.

The external environment estimating unit 14 estimates an external environment representing the actual external environment based on environmental information representing the external environment. The external environment estimation unit 14 generates external environment information B10 as an estimation result, and outputs it to the determination unit 15.

The external environment information B10 may include external brightness information B10a (second brightness information) indicating the actual brightness of the outside. The external brightness information B10a corresponds to the brightness of an image actually photographed outside. The external environment estimation unit 14 estimates external brightness information B10a using the illuminance information b11 acquired from the illuminance sensor 5.

Specifically, when the interphone button 2 is pressed, the external environment estimation unit 14 acquires the illuminance information b11 from the illuminance sensor 5. The external environment estimation unit 14 estimates external brightness information B10a based on the illuminance information b11. The external environment estimation unit 14 may correct the illuminance information b11 as appropriate depending on the installation position of the illuminance sensor 5, etc., and estimate the external brightness information B10a.

Further, the external environment estimating unit 14 may acquire the second video B1 shot during a predetermined period from the video recording unit 4 and estimate the external brightness information B10a. For example, the external environment estimating unit 14 acquires the second video B1 for a predetermined period before the visitor appears from the video recording unit 4. This is because it is assumed that the video taken during this period is not a fake video but an actual external video. The external environment estimation unit 14 estimates the external brightness information B10a by estimating the current actual brightness of the external video using the video.

Further, the external environment estimating unit 14 may acquire the environmental information b12 from the environmental information DB6 and estimate the external brightness information B10a using the external weather information B10b included in the environmental information b12.

Specifically, the external environment estimation unit 14 refers to a predetermined table stored in the environmental information DB 6, obtains the brightness corresponding to the external weather information B10b around the residence, and estimates the external brightness information B10a. The predetermined table includes information that associates weather with brightness.

The method is not limited to these, and the external environment estimation unit 14 may estimate the external brightness information B10a using other methods. The external environment estimating unit 14 can estimate the external brightness information B10a using various pieces of information that show the correlation between the weather around the residence and the actual brightness outside. By estimating the actual brightness of the outside, the external environment estimating unit 14 can obtain the correct brightness of the image photographed by the photographing unit 3.

Further, the external environment information B10 may include external weather information B10b indicating the actual weather outside. The external environment estimating unit 14 can acquire the external weather information B10b from the environmental information DB 6, include the external weather information B10b in the external environment information B10, and output it to the determining unit 15. The external weather information B10b may include, for example, information such as clear weather, cloudy weather, rain, thunder, snow, or strong wind.

As described above, the external environment estimating unit 14 uses the illuminance sensor 5 and the environmental information DB 6 to output the actual external environment when the intercom button 2 is pressed to the determining unit 15 as the external environment information B10. Can be done.

Note that the external environment estimation unit 14 may generate the external environment information B10 using either the illuminance sensor 5 or the environment information DB 6, or may use both of them to generate the external environment information B10. Therefore, the external environment information B10 may include either the external brightness information B10a or the external weather information B10b, or may include both. In addition to the illuminance information b11 acquired from the illuminance sensor 5, the external environment estimation unit 14 may generate the external environment information B10 by supplementarily using the environmental information b12 acquired from the environmental information DB 6. The determination unit 15 can more accurately determine a false image by making the determination using both the external brightness information B10a and the external weather information B10b.

The determination unit 15 compares the first video A1 and the second video B1, and if there is a change greater than a predetermined value, the determination unit 15 determines that the first video A1 is a fake video (fake image) that is not a video actually shot outside. A determination process is performed to determine that there is. For example, the determination unit 15 performs the determination process when a call on the interphone button 2 is detected.

The change greater than a predetermined value may indicate, for example, movement of a person or object included in the second video B1. For example, a change greater than a predetermined value may include a person or thing appearing, a person or thing disappearing, a person becoming a different person, or a thing becoming another thing. The determination unit 15 compares the first video A1 and the second video B1, and determines whether the arrangement of people or objects shown has changed by a predetermined amount or more. Further, the determination unit 15 may determine whether the appearance of the person or object in the image has changed by a predetermined amount or more. The determination unit 15 may make the determination using both a change in arrangement and a change in appearance.

The determining unit 15 determines whether there is a change greater than a predetermined value in the first video A1 and the second video B1 by comparing the video scene A1a of the first video A1 and the video scene B1a of the second video B1. compare. For example, the determination unit 15 acquires image information from the video scenes A1a and B1a. The image information may include, for example, brightness information and color information of each of the video scenes A1a and B1a. The determining unit 15 determines the difference between the image information of the video scene A1a and the image information of the video scene B1a. The determination unit 15 determines that the first video A1 is a fake video when the difference is equal to or greater than a predetermined value.

Further, suppose that a visitor holds up a fake video to the photographing unit 3 using a mobile terminal such as a smartphone. In this case, when holding the mobile terminal over the photographing section 3, it is assumed that the photographing section 3 will be covered and become dark. Utilizing this, the determining unit 15 may determine that the first image A1 is a false image when the brightness of the first image A1 changes by more than a predetermined value. Further, the determination unit 15 may perform the determination using a change in the frame rate that occurs in response to the change in brightness. When the frame rate of the first video A1 changes by more than a predetermined value, the determining unit 15 determines that the brightness of the first video A1 changes by more than a predetermined value, and determines that the first video A1 is a fake video.

Further, the determination unit 15 acquires video environment information A10 from the video environment estimation unit 13 and external environment information B10 from the external environment estimation unit 14. The video environment information A10 may include video brightness information A10a and video weather information A10b. Furthermore, the external environment information B10 may include external brightness information B10a and external weather information B10b.

The determination unit 15 may perform the determination process using the video environment information A10 and the external environment information B10. For example, the determining unit 15 compares the video brightness information A10a and the external brightness information B10a, and determines that the first video A1 is a fake video according to the comparison result. The determination unit 15 determines that the first video A1 is a false video if the difference between the video brightness information A10a and the external brightness information B10a is not within a predetermined range. The predetermined range may be set as appropriate.

By doing so, when the first image A1 is darker or brighter than the actual outside, the determination unit 15 can determine that the first image A1 is a false image.

Note that the determining unit 15 may determine whether the video is a fake video by taking the current time into consideration. For example, suppose that a visitor holds up a fake video shot during the day to the imaging unit 3 during the night time period. In this case, the first video A1 includes an image that is brighter than the actual outside. The determination unit 15 determines whether or not the video brightness information A10a of the first video A1 indicates a brightness range corresponding to the time zone, and determines that the first video A1 is a false video according to the determination result. .

Further, the determination unit 15 may compare the video weather information A10b and the external weather information B10b, and determine that the first video A1 is a fake video according to the comparison result. The determination unit 15 determines that the first video A1 is a fake video when the video weather information A10b and the external weather information B10b indicate different weather.

For example, suppose that a visitor holds up a false image taken on a sunny day to the photographing unit 3 when it is actually raining near the residence. In this case, the video weather information A10b indicates "sunny" and the external weather information B10b indicates "rainy." Since the video weather information A10b and the external weather information B10b indicate different weather, the determination unit 15 can determine that the first video A1 is a fake video. Thereby, the determination unit 15 can evaluate whether the brightness of the first video A1 is appropriate based on weather information that can be obtained from the Internet or the like.

Note that the determination unit 15 may perform the process of determining whether or not the first video A1 described above is a fake image when a person other than a pre-registered person is included in the first video A1. .

For example, the resident uses an information terminal or the like to register in the registrant DB 16 in advance non-judgment targets who are not subject to the fake video determination process. The non-judgment target person is, for example, a resident, a relative of the resident, or a friend. Furthermore, a delivery person who delivers packages or the like may be included in the non-judgment target persons. The determination system 200 may analyze the second video B1 and automatically register delivery personnel who have delivered a predetermined number of times or more as non-judgment candidates.

The registrant DB 16 stores, for example, a registrant ID that identifies a registrant and a face image of the registrant in association with each other. The facial image may include information on feature points of the registrant's face. Thereby, the determining unit 15 can identify whether the person included in the first video A1 is a registrant.

The determination unit 15 does not perform the determination process on non-judgment target persons registered in the registrant DB 16, but performs the determination process on persons other than the non-judgment target people. By doing so, the determination unit 15 does not perform the false image determination process on a person who is unlikely to cause any disadvantage to the resident, but performs the determination process on other people. Can be done.

Further, the registrant DB 16 may register a person with specific characteristics as a specific person. The specific person is a person who can be a target of caution. The specific feature is, for example, information about a person's clothes or belongings. Information regarding clothing includes, for example, whether the user wears a uniform, a suit, or a company badge. By registering such characteristics, for example, when the first video A1 includes a person such as a police officer or an insurance agent, the determination unit 15 can determine the person in the first video A1 as a specific person. can do. Thereby, the determination unit 15 identifies impersonation of a police officer, impersonation of an insurance salesman, and the like.

Further, the registrant DB 16 may register information such as persons who are targets of caution in the surrounding area, persons who are wanted, and the like. As a result, if the person holding up the false image is a person to be monitored, appropriate measures can be taken.

In addition, in the above description, a case has been described in which a pre-registered person is not subject to the determination process, but the registrant DB 16 may register a person to be determined in advance. The person to be judged is a person who is to be judged whether or not the image is a fake image. The person to be judged may be a specific individual, or a person wearing a specific outfit as described above.

The determination unit 15 controls the video recording unit 4 according to the determination result of whether the first video A1 is a fake video. If the first video A1 is determined to be a false video, the determination unit 15 controls the video recording unit 4 to record a video from a predetermined time ago. For example, the determination unit 15 records a video from a predetermined time period before the time T1 when the visitor who caused the false video to be photographed pressed the intercom button 2. Alternatively, the determining unit 15 may determine the timing at which the visitor enters the photographing range of the photographing unit 3 and the visitor is captured, and record the video from a predetermined time before the timing.

Further, for example, if a visitor holds up a mobile terminal without entering the photographing range, the determination unit 15 determines the timing when the photographing unit 3 is covered and becomes dark, and detects the timing from a predetermined time before the said timing. The video may be recorded.

In this way, when the first video A1 is determined to be a fake video, the determination unit 15 can cause the video recording unit 4 to record a video from before the timing when the fake video was shot. Thereby, the video recording unit 4 can record a visitor who attempts to illegally open the door using a fake video.

Furthermore, when the first video A1 is determined to be a false video, the determination unit 15 may instruct the locking control unit 7 to lock the entrance door, close the window, or lock the window. By locking these all at once, it is possible to prevent visitors from entering the residence.

Further, if the first video A1 is determined to be a false video, the determination unit 15 may control the notification unit 8 to notify the resident or the like. For example, when determining that the first video A1 includes a specific person, the determination unit 15 may notify the resident's information terminal or the like of this fact. This allows residents and the like to know that there is a person to be wary of outside the door. Furthermore, when a resident or the like is out of the house, it is possible to grasp such a situation while away.

Further, in such a case, the determination unit 15 may use the notification unit 8 to sound a siren. By doing this, appropriate measures can be taken in the event that a visitor attempts to open the door using a false image. Furthermore, if the visitor is a specific person, this can be appropriately notified.

Moreover, the determination unit 15 may report to a police station or the like when a person to be cautioned is shown in the first image A1. As a result, if the person holding up the false image is a person to be monitored, appropriate measures can be taken.

Note that the configuration of the determination system 200 shown in FIG. 1 is only an example. The determination system 200 may be configured using a device in which a plurality of configurations are integrated. For example, the functions of the determination device 10 and the functions other than the determination device 10 may be integrated into the same device. Further, for example, each functional unit in the determination device 10 may be realized using a plurality of devices. For example, in the example of FIG. 1, the first video acquisition section 11, the second video acquisition section 12, the video environment estimation section 13, the external environment estimation section 14, the determination section 15, and the registrant DB 16 are shown inside the determination device 10. However, some of these may be distributed.

(Processing of determination device 10)
Next, the processing performed by the determination device 10 will be described with reference to FIGS. 2 to 4. 2 to 4 are flowcharts showing the processing performed by the determination device 10.

First, with reference to FIG. 2, the overall flow of processing performed by the determination device 10 will be described. FIG. 2 is a flowchart showing the overall flow of processing performed by the determination device 10. Note that in the determination system 200, the imaging unit 3 constantly performs imaging and outputs the first video A1 to the video recording unit 4. The video recording unit 4 temporarily records the video as a second video B1.

The first video acquisition unit 11 determines whether there is a call from the interphone button 2 (S11). If there is no call (NO in S11), the first video acquisition unit 11 waits until there is a call. When there is a call (YES in S11), the first video acquisition unit 11 acquires the first video A1 from the imaging unit 3 (S12). The first video acquisition unit 11 extracts the person included in the first video acquisition unit 11 and outputs the extraction result to the determination unit 15.

The determining unit 15 refers to the registrant DB 16 and determines whether the extracted person is a registrant (S13). The registrant is a person who is not subject to the determination process, and is, for example, a family member of the resident. If the person is a registrant (YES in S13), the determination unit 15 ends the process. If the person is not a registrant (NO in S13), the determination unit 15 acquires the second video B1 from the video recording unit 4 (S14).

The determination unit 15 compares the first video A1 and the second video B1 and determines whether the first video A1 is a fake video (S15). The determination unit 15 determines that the first video A1 is a false video if there is a change greater than a predetermined value between the first video A1 and the second video B1. The determination unit 15 may perform the fake video determination process in step S15 by selecting from a plurality of determination methods, or may perform the process of determining a false video by using a plurality of determination methods in combination.

Here, the first determination method will be explained with reference to FIG. 3. FIG. 3 is a flowchart showing the first determination method.

The determination unit 15 compares the video scene A1a of the first video A1 and the video scene B1a of the second video B1 (S21). The determining unit 15 determines whether the arrangement of people and objects shown in each of the video scene A1a and the video scene B1a has changed by a predetermined amount or more (S22). Examples of changes greater than a predetermined value include a person or thing appearing, a person or thing disappearing, a person becoming a different person, or a thing becoming another thing. Alternatively, the determination unit 15 may determine whether there is a change greater than a predetermined value using a change in frame rate between the first video A1 and the second video B1.

If it is determined that there is no change greater than the predetermined value (NO in S22), the determination unit 15 determines that the first video A1 is not a false video (S23). Further, when it is determined that there has been a change greater than a predetermined value (YES in S22), the determination unit 15 determines that the first video A1 is a false video (S24).

As a result, the determination unit 15 determines whether a person who is not shown in the second video B1 suddenly appears in the first video A1, or if the frame rate changes because the imaging unit 3 is covered with an information terminal. 1 video A1 can be determined to be a fake video.

Next, the second determination method will be described with reference to FIG. 4. FIG. 4 is a flowchart showing the second determination method.

The video environment estimation unit 13 acquires the first video A1, and estimates the video environment information A10 based on the first video A1 (S31). The video environment information A10 may include video brightness information A10a indicating the external brightness corresponding to the first video A1. The image brightness information A10a corresponds to the brightness of the first image A1.

Next, the external environment estimation unit 14 acquires the illuminance information b11 or the environment information b12 (S32). The external environment estimation unit 14 acquires illuminance information b11 from the illuminance sensor 5. Alternatively, the external environment estimation unit 14 acquires the environment information b12 from the environment information DB 6. Note that the external environment estimation unit 14 may acquire both the illuminance information b11 and the environment information b12.

The external environment estimation unit 14 estimates external environment information B10 based on the illuminance information b11 or the environment information b12 (S33). The external environment information B10 may include external brightness information B10a indicating the actual brightness of the outside. The external brightness information B10a corresponds to the brightness of an image actually photographed outside. For example, the external environment estimation unit 14 estimates the external brightness information B10a using the illuminance information b11.

The external environment estimation unit 14 may estimate the external brightness information B10a using the external weather information B10b included in the environmental information b12. The external environment estimation unit 14 refers to a predetermined table stored in the environmental information DB 6 and estimates the external brightness information B10a by acquiring the brightness corresponding to the external weather information B10b.

The determination unit 15 compares the video environment information A10 and the external environment information B10 (S34). The determination unit 15 determines whether there is a difference greater than a predetermined value between the video environment information A10 and the external environment information B10 (S35).

If it is determined that there is no difference greater than the predetermined value (NO in S35), the determining unit 15 determines that the first video A1 is not a fake video (S36). Further, when it is determined that there is a difference greater than a predetermined value (YES in S35), the determination unit 15 determines that the first video A1 is a fake video (S37).

Thereby, the determination unit 15 can determine that the first video A1 is a false video if the weather or brightness in the first video A1 does not correspond to the actual external weather or brightness.

Returning to FIG. 2, the explanation will be continued. As a result of the above-described determination, the determination unit 15 determines whether the first video A1 is a fake video (S16). If it is determined that the video is not a fake video (NO in S16), the process ends. If it is determined that the image is a fake video (YES in S16), the determination unit 15 controls the notification unit 8 to make a notification (S17). The notification may be, for example, outputting a warning sound, displaying a warning message, notifying a resident's information terminal, or reporting to the police. Further, when determining that the first video A1 includes a specific person, the determining unit 15 may notify the resident's information terminal, the police, etc. of this fact.

Further, the determination unit 15 controls the locking control unit 7 to lock the residence (S18). For example, the determination unit 15 instructs the locking control unit 7 to lock the entrance door, close and lock the windows, etc.

Further, the determination unit 15 controls the video recording unit 4 to record video for a predetermined period (S19). For example, the determination unit 15 does not delete the video of a predetermined time period before the shooting of the false video starts from the video recording unit 4, but retains the video. Thereby, the video recording unit 4 can record and hold the video from the timing before the fake video was shot.

Note that the order of the above-described processes may be changed as appropriate. For example, the determination unit 15 may perform the processes of steps S17 to S19 in a different order. Furthermore, the determination unit 15 may not perform part or all of the processing in steps S13 and S17 to S19. That is, the determination unit 15 may perform the determination process on all visitors. Further, the determination unit 15 does not need to perform notification or locking.

As explained above, in the determination system 200 according to the present embodiment, the determination device 10 is able to detect the first video captured by the imaging unit 3 and the second video captured by the same imaging unit 3 before the first video. Get the video and. The determination device 10 compares the first video and the second video, and determines that the first video is a fake video that is not an actual video taken outside if there is a change greater than a predetermined value.

For example, the determination device 10 compares each scene of the first video and the second video to determine whether or not the image information has changed by a predetermined amount or more. Thereby, the determination device 10 can determine that the first video is a false video when the appearance of a person or object that does not exist in the second video is detected.

Further, the determination device 10 estimates first brightness information indicating the external brightness corresponding to the first image and second brightness information indicating the actual external brightness, and compares the two to determine if the false image is displayed. Make a judgment. Thereby, the determination device 10 can accurately determine false images using information obtained from the illuminance sensor and weather information on the Internet.

By doing so, the determination system 200 can appropriately determine whether a video taken outside the predetermined area is genuine or not, and can immediately identify a malicious visitor. Furthermore, when such a situation occurs, the determination device 10 locks the front door and windows to block all entrances to the residence. This can prevent visitors from entering your home. Furthermore, by using an alarm such as a siren, the determination device 10 can repel those who attempt to illegally invade. Since the determination device 10 can notify residents and the police, it is possible to improve the security of the residence.

(Hardware configuration example)
Each functional component of the determination device 10 described above may be realized by hardware that implements each functional component (e.g., a hardwired electronic circuit, etc.), or by a combination of hardware and software (e.g., a hardwired electronic circuit, etc.). (e.g., a combination of an electronic circuit and a program that controls it). For example, the present disclosure can also implement arbitrary processing by causing a CPU (Central Processing Unit) to execute a computer program.

The program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more of the functions described in the embodiments. The program may be stored on a non-transitory computer readable medium or a tangible storage medium. By way of example and not limitation, computer readable or tangible storage media may include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD -Includes ROM, digital versatile disc (DVD), Blu-ray disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disc storage or other magnetic storage device. A program may be transmitted on various types of transitory computer readable media or communication media. By way of example and not limitation, transitory computer-readable or communication media includes electrical, optical, acoustic, or other forms of propagating signals.

Note that the present disclosure is not limited to the above embodiments, and can be modified as appropriate without departing from the spirit. For example, in the above description, it is assumed that a person is shown in the first video A1, but the present invention is not limited to this. The determination device 10 may perform the above-described determination process even when no person is shown in the first video A1. This can prevent residents and the like from opening the door to check what is going on outside.

This disclosure includes matters that contribute to the realization of the SDGs of ``Creating sustainable cities'' and contribute to the safety and security of public facilities.

1 Door scope 2 Intercom button 3 Photographing section 4 Video recording section 5 Illuminance sensor 6 Environmental information DB
7 Lock control section 8 Notification section 10 Determination device 11 First image acquisition section (first image acquisition section)
12 Second image acquisition unit (second image acquisition unit)
13 Video environment estimation section 14 External environment estimation section 15 Judgment section 16 Registrant DB
200 Judgment system A1 1st video (1st image)
A1a Video scene A10 Video environment information A10a Video brightness information (first brightness information)
A10b Video weather information b11 Illuminance information b12 Environmental information B1 Second video (second image)
B1a Video scene A1a
B10 External environment information B10a External brightness information (second brightness information)
B10b External weather information

Claims (5)

a first image acquisition section that acquires a first image photographed by a photographing section that photographs the outside of a predetermined area;
a second image acquisition unit that acquires a second image photographed by the photographing unit before photographing the first image;
a determination unit that compares the first image and the second image, and determines that the first image is a false image that is not an image actually photographed outside if there is a change greater than a predetermined value; A determination device comprising: a first estimation unit that estimates first brightness information indicating external brightness corresponding to the first image based on the first image;
a second estimation unit that estimates second brightness information indicating the actual brightness of the outside based on external environment information indicating the external environment;
Furthermore,
The determination device according to claim 1, wherein the determination unit compares the first brightness information and the second brightness information, and determines that the first image is a false image according to a comparison result. The determination device according to claim 1 or 2, wherein the determination unit determines whether the first image is a false image when a person other than a pre-registered person is included in the first image. . The computer is
a first image acquisition step of acquiring a first image photographed by a photographing unit photographing the outside of the predetermined area;
a second image acquisition step of acquiring a second image photographed by the photographing unit before photographing the first image;
a determination step of comparing the first image and the second image, and determining that the first image is a false image that is not an image actually photographed outside if there is a change greater than a predetermined value; Execute the judgment method. a first image acquisition step of acquiring a first image photographed by a photographing unit photographing the outside of the predetermined area;
a second image acquisition step of acquiring a second image photographed by the photographing unit before photographing the first image;
a determination step of comparing the first image and the second image, and determining that the first image is a false image that is not an image actually photographed outside if there is a change greater than a predetermined value; A program that causes a computer to execute.

Images