KR102545534B1 - Doorbell, security system including doorbell apparatus, and method of operating the same - Google Patents

Abstract

A doorbell device according to an embodiment of the present invention includes a camera that acquires a first image of an object in the first area and a second image of an object in the second area; an image processing unit detecting the object in the first image and identifying the object in the second image; a storage unit for storing the second image corresponding to the identified object; and an alarm unit activating an alarm signal according to whether the identified object corresponds to a predetermined object, wherein the second area is an area closer to the camera than the first area, and the image processing unit It consists of a first image processing unit that detects and stores a first image and a second image processing unit that identifies and stores objects in the second image.

Description

Doorbell, security system having the same and method of operating the same

Embodiments of the present invention relate to a doorbell device, and more particularly, to a doorbell capable of reducing an object detection error when an object is captured by a camera of the doorbell device, a security system having the same, and an operation method thereof.

In general, a camera may be installed in the doorbell device so that the visitor can be identified by an image of an environment near the door from the inside when the visitor presses the bell.

Recently, a photo of a visitor photographed by a camera of a doorbell device can be transmitted as a push message to a user's terminal (eg mobile phone, etc.), and the user receives the push message and makes a video call with the visitor through the doorbell device when necessary. can do

However, in the conventional case, since the camera of the doorbell uses an image-based motion detection technology, an error in which a change in illuminance is detected as a motion may occur. Therefore, when a user receives a push message and connects to a video call, a case may occur in which a visitor is not identified as an empty video.

A doorbell according to an embodiment of the present invention and a security system having the same can reduce errors in detecting and imaging an object. In one example, an object may be detected at a distance from the camera, but may not be identified and imaged until the object is detected at a distance closer to the camera. That is, a doorbell according to an embodiment of the present invention, a security system having the same, and an operating method thereof detect objects at different distances, more accurately identify objects detected at a close distance, and provide the information, thereby detecting objects. Errors can be reduced, and accurate and diverse information about the detected visitor can be provided to the user.

In order to achieve the above object, a security system according to an embodiment of the present invention includes a doorbell detecting an object in a first area and identifying an object in a second area; a user terminal receiving information corresponding to the identified object from the doorbell; and a communication network providing one or more communication channels between the user terminal and the doorbell, wherein the doorbell acquires a first image of an object in the first area and a second image of an object in the second area. camera; an image processing unit detecting the object in the first image and identifying the object in the second image; a storage unit for storing the second image corresponding to the identified object; and an alarm unit activating an alarm signal according to whether the identified object corresponds to a predetermined object, wherein the second area is an area closer to the camera than the first area, and the image processing unit It consists of a first image processing unit that detects and stores a first image and a second image processing unit that identifies and stores all objects in the second image.

The security system may further include a server that collects and analyzes event information to generate state information and transmits the state information to the user terminal, wherein the state information is a situation in which the object is detected and/or identified. It can be provided in a message format suitable for

The server may classify the event information into predetermined periods, summarize the event information, and transmit the summarized event information to the user terminal.

The user terminal may access the storage unit, select a specific event of the storage unit, and store the specific event in separate categories.

The doorbell transmits a first signal to the camera to capture the first image when the object is located in the first area, and transmits the second image when the object is located in the second area. It may further include a motion detector that transmits a second signal to the camera to capture.

The doorbell transmits a first signal to the first image processing unit to detect an object in the first image when the object is located in the first area, and when the object is located in the first area, the doorbell transmits a first signal to the first image processing unit. A motion detector for transmitting a second signal to the second image processing unit to identify objects in the two images may be further included.

The first and second image processing units may individually detect and/or identify the object by applying different image processing algorithms.

A doorbell device according to an embodiment of the present invention includes a camera that acquires a first image of an object in the first area and a second image of an object in the second area; an image processing unit detecting the object in the first image and identifying the object in the second image; a storage unit for storing the second image corresponding to the identified object; and an alarm unit activating an alarm signal according to whether the identified object corresponds to a predetermined object, wherein the second area is an area closer to the camera than the first area, and the image processing unit It consists of a first image processing unit that detects and stores a first image and a second image processing unit that identifies and stores objects in the second image.

A motion detector for determining whether the object is in the first area or the second area may be further included.

The motion detector transmits a first signal to the camera to capture the first image when the object is located in the first area, and to capture the second image when the object is located in the second area. A signal may be transmitted to the camera.

The motion detector transmits a first signal to the first image processing unit to detect an object in the first image when the object is located in the first area, and transmits a first signal to the second image processing unit when the object is located in the first area. A second signal may be transmitted to the second image processing unit to identify an object in the image.

The image processing unit may identify all objects in the second image.

The first and second image processing units may individually detect and/or identify the object by applying different image processing algorithms.

The image processing unit may divide the image acquired by the camera into first and second parts, and apply different algorithms to the image data of the first and second parts.

The first image processing unit may apply an object detection algorithm to a first portion corresponding to the first area, and the second image processing unit may apply an object identification algorithm to a second portion corresponding to the second area. there is.

The second part may be a central part of an image acquired by the camera.

The doorbell device includes a microphone installed outside the door; and a sound processing unit that detects a sound event received from the microphone and stores an image captured by the camera in the storage unit when the sound event is detected.

A method of operating a security system according to an embodiment of the present invention includes capturing a first image of an object in a first area; detecting the object in the first image; capturing a second image of an object in a second area closer to the first area; identifying the object in the second image; storing the second image corresponding to the identified object; and transmitting information corresponding to the identified object to a user terminal.

The method of operating the security system may further include classifying the information into predetermined periods, summarizing the information for each predetermined period, and transmitting the summary to the user terminal.

The operating method of the security system may include comparing the second image including the identified object with pre-stored images; and transmitting information corresponding to the same identified object as the pre-stored images to the user terminal.

According to the doorbell according to an embodiment of the present invention and the security system having the same, errors in detecting and imaging an object can be reduced. In one example, an object may be detected at a distance from the camera, but may not be identified and imaged until the object is detected at a distance closer to the camera.

That is, in the case of an embodiment of the present invention, by detecting each object at different distances, more accurately identifying an object detected at a close distance and providing the information, object detection errors are reduced, and information about the detected visitor is provided to the user. Able to provide accurate and diverse information.

1 is a block diagram schematically illustrating a security system according to an embodiment of the present invention;
2 is a flowchart schematically illustrating the operation of the security system shown in FIG. 1;
3 is a block diagram showing a doorbell device according to an embodiment of the present invention;
4 is a diagram for explaining an example of a screen of a user terminal according to an embodiment of the present invention.
5 is a block diagram showing components of the processor shown in FIG. 3;
6A and 6B are diagrams for explaining an example of driving the first image processing unit and the second image processing unit shown in FIG. 5;
7 is a diagram for explaining an example of video event detection according to an embodiment of the present invention;
8 is a diagram schematically illustrating a security system according to another embodiment of the present invention.
9 is a flowchart schematically illustrating the operation of the security system shown in FIG. 8;
10 and 11 are flowcharts illustrating the operation of a security system according to another embodiment of the present invention.

The information described in the technical background of the above invention is only to help the understanding of the background art of the technical idea of the present invention, and therefore it can be understood as the prior art known to those skilled in the art of the present invention. does not exist.

In the following description, for purposes of explanation, numerous specific details are set forth to facilitate an understanding of various embodiments. It is evident, however, that the various embodiments may be practiced without these specific details or in one or more equivalent manners. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the understanding of the various embodiments.

In the drawings, the size or relative size of layers, films, panels, regions, etc. may be exaggerated for clarity. Also, like reference numerals denote like elements.

Throughout the specification, when an element or layer is described as "connected" to another element or layer, this is not only directly connected, but also indirectly connected with another element or layer intervening therebetween. Including case However, if a part is described as being "directly connected" to another part, it will mean that there are no other components between that part and the other part. "At least one of X, Y, and Z" and "at least one selected from the group consisting of X, Y, and Z" means one X, one Y, one Z, or two or more of X, Y, and Z Any combination of more (eg, XYZ, XYY, YZ, ZZ) will be understood. Here, "and/or" includes any combination of one or more of the elements.

Here, although terms such as first, second, etc. may be used to describe various elements, elements, regions, layers, and/or sections, such elements, elements, regions, layers, and/or or sections are not limited to these terms. These terms are used to distinguish one element, element, region, layer, and/or section from another element, element, region, layer, and/or section. Thus, a first element, element, region, layer, and/or section in one embodiment may be referred to as a second element, element, region, layer, and/or section in another embodiment.

Spatially relative terms such as "below", "above", etc. may be used for descriptive purposes, thereby describing the relationship of one element or feature to another element(s) or feature(s) as shown in the figures. do. This is only used to indicate the relationship of one component to another component on the drawing, and does not mean an absolute position. For example, if the device shown in the figures is turned upside down, elements depicted as being “below” other elements or features will be positioned in a direction “above” the other elements or features. Thus, in one embodiment, the term “below” may include both directions of up and down. In addition, the device may be in other orientations (eg, rotated 90 degrees or in other orientations), and such spatially relative terms used herein are interpreted accordingly.

Terminology used herein is for the purpose of describing specific embodiments and not for the purpose of limitation. Throughout the specification, when a certain component is said to "include", it means that it may further include other components without excluding other components unless otherwise stated. Unless otherwise defined, terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

1 is a block diagram schematically illustrating a security system according to an embodiment of the present invention, and FIG. 2 is a flowchart schematically illustrating the operation of the security system shown in FIG. 1 .

Referring to FIG. 1, the security system according to an embodiment of the present invention includes a doorbell device 10 and a user terminal 30, and the doorbell device 10 connects the user terminal 30 through a network 20. ) can communicate with.

The network 20 is a short-distance wireless communication network such as Zigbee, Bluetooth, Radio Frequency Identification (RFID), NFC, infrared communication, and 3G, 4G (LTE), Wi-Fi, Wibro , a mobile communication network such as Wimax, a wireless Internet communication network, a Sub 1 GHz band communication, and/or a wired communication network.

In general, the doorbell device 10 outputs an alarm sound from a video phone installed inside when a visitor presses a button or a bell 109 of the doorbell device 10, and the user inside the video phone You can identify visitors by seeing and hearing their faces and voices.

The doorbell device 10 according to an embodiment of the present invention can communicate with the user terminal 30 in real time using a wireless network in case the user is not in the space where the doorbell device 10 is installed and is in a remote location. can Accordingly, when a visitor enters the imaging area of the doorbell device 10 or presses the bell 109, functions such as saving an image or providing an absent alarm function may be provided.

The doorbell device 10 may interwork with at least one registered user terminal 30 using wireless communication.

Referring to FIG. 2 , the doorbell device 10 may detect a motion event from an image acquired by a camera (S11). In one embodiment, the doorbell device 10 may detect and/or identify an object by applying two or more different image processing algorithms to each region image. When a motion event is detected, the doorbell device 10 may transmit a motion event push to the user terminal 30 (S12).

The doorbell device 10 may detect a sound event from sound acquired by a microphone (S13). The doorbell device 10 may detect a sound event by detecting an abnormal sound in the surroundings. When a sound event is detected, the doorbell device 10 may transmit a sound event push to the user terminal 30 (S14).

The doorbell device 10 may transmit at least one of a corresponding image, information of the identified object OB, and information of an alarm sound set in the object OB to the user terminal 30 together with the event push. The alarm sound information may include alarm sound discrimination data such as a title, an identifier, and an index of an alarm sound promised between the doorbell device 10 and the user terminal 30 .

The user terminal 30 receives motion and/or sound event pushes, and the user can grasp a situation around the doorbell device 10 through this. In one embodiment, the user terminal 30 receives information on the identified object (OB), selects a corresponding alarm sound from alarm sound information for each object (OB) stored in the user terminal 30, and generates a corresponding sound source. can output

In another embodiment, the user terminal 30 may receive alarm sound information and output a sound source corresponding to the received alarm sound information. When the doorbell device 10 detects a bell activation event (S15), it may transmit a bell event push to the user terminal 30 (S16). The user may select a video mode and/or a sound mode on the user terminal 30 to conduct a video call and/or an audio call with a visitor in real time.

3 is a block diagram showing a doorbell device according to an embodiment of the present invention.

Referring to FIG. 3 , a doorbell device 10 according to an embodiment may include a processor 100, a sensor unit 101, a communication unit 103, a storage unit 105, and a power supply unit 107. The sensor unit 101 may include at least one sensor built into the doorbell device 10 or installed inside (indoor) and/or outside (outdoor) a door where the doorbell device 10 is installed. . The at least one sensor may include a camera 112 , a microphone 114 and a motion detector 116 .

The camera 112 may include an image sensor, a lens, and a filter. The camera 112 is installed inside and/or outside the door, and can capture indoor and/or outdoor situations. The camera 112 may capture an image of a predetermined area around the doorbell device 10 . According to an embodiment of the present invention, the camera 112 may capture an image of an object in a first area and/or an object in a second area different from the first area.

For example, the second area is an area closer to the camera 112 (ie, the doorbell device) than the first area. The first area and the second area may be set according to the performance of the camera 112 and the external environment.

The first area may be a range of a distance capable of detecting motion of an object or whether the object is a person from an image captured by the camera 112 . The second area may be a range of a distance at which an object can be identified, such as recognizing a face of an object from an image captured by the camera 112 .

For example, the first area may be an area at a distance of 10 m or 5 m from the camera 112 , and the second area may be an area at a distance of 5 m or 3 m from the camera 112 .

The microphone 114 is installed on the outside of the door and can sense outdoor sound. The sensor unit 101 may further include various sensors capable of detecting external intrusion and/or danger or accidents, such as a gas sensor, a temperature sensor, and an illuminance sensor.

For example, motion of an object approaching the camera 112 may be detected using the motion detector 116 and distance information between the object and the camera may be obtained. The motion detector 116 may include an infrared rays sensor. When an infrared sensor is used, the incident position of reflected infrared rays changes according to near and far distances, and based on this, the distance between the object and the camera can be calculated. It is possible to determine whether an object is in the first area or the second area through the motion detector 116 . The motion detector 116 transmits a first signal to the camera 112 and/or the processor 100 when it determines that the object is in a first area, and transmits a second signal when it determines that the object is in a second area. A signal may be transmitted to the camera 112 and/or the processor 100 .

The communication unit 103 is a short-range wireless communication such as Zigbee, Bluetooth, Radio Frequency Identification (RFID), NFC, infrared communication, and 3G, 4G (LTE), Wi-Fi, Wibro, and WiMAX (Wimax), etc., or wireless Internet communication may be performed. In addition, the communication unit 103 may perform Sub 1 GHz band communication and/or wired communication. The communication unit 103 may be connected to the user terminal 30 through wired or wireless communication.

The storage unit 105 may include a memory 105a and a database 105b. The memory 105a may store programs for processing and control of the processor 100 or may store input or output data. In the database 105b, video events in which objects located in the second area are photographed may be stored according to a user's selection. For example, the objects may be classified and stored in a database by groups such as family and friends. The storage unit 105 may include an internal memory and/or an external storage medium such as an SD card. The storage unit 105 may include a web storage that performs a storage function on the Internet, a cloud server, and the like.

The user can connect to the storage unit 105 through the user terminal 30 and search for and reproduce stored motion and/or sound that matches conditions for each event and each time.

4 is a diagram illustrating an example of a screen of a user terminal according to an embodiment of the present invention. Referring to FIG. 4 , a user may select video events in which a specific object is photographed through the screen 300 of the terminal and store them in the database 105b.

That is, the user can connect to the storage unit 105 through the user terminal 30 and classify specific video events into separate categories and store them in the database 105b. In this case, the category may be divided into a plurality of groups such as family and friends, and when an object included in the group is later visited and recognized as the same object, it may be provided as a separate event to the user terminal. The stored image event of each object may include not only the front side of the object but also the side view.

The power supply unit 107 may convert AC or DC power applied from an external power source into power required for operation of each component and supply the converted power. The power supply unit 107 may include an auxiliary power source such as a rechargeable battery.

Processor 100 may be implemented with any number of hardware or/and software components that perform specific functions. For example, the processor 100 may refer to a data processing device embedded in hardware having a physically structured circuit to perform functions expressed by codes or instructions included in a program.

The processor 100 controls the overall operation of the doorbell device 10, and performs functions such as exchanging control signals with the components or processing data to control the operation of each component.

FIG. 5 is a block diagram illustrating components of the processor shown in FIG. 3 . 6A and 6B are diagrams for explaining an example of driving the first image processing unit and the second image processing unit shown in FIG. 4 .

Referring to FIG. 5 , the processor 100 may include an image processing unit 111 , a sound processing unit 113 and an alarm unit 115 .

The image processing unit 111 may include an image codec and may process an image captured by the camera 112 . The image processing unit 111 includes a first image processing unit 111a that processes an image event of an object acquired in a first area and a second image processing unit that processes an image event of an object acquired in a second area ( 111b) may be included. That is, the first image processing unit 111a may detect and store the first image, and the second image processing unit 111b may identify and store all objects of the second image.

In one embodiment, the first and second image processing units 111a and 111b may individually detect and/or identify objects by applying different image processing algorithms. For example, the first image processing unit 111a may apply an object detection algorithm such as a motion detection (eg, motion detection (MD) algorithm) and a human detection (eg, human detection (HD) algorithm). In addition, the second image processing unit 111b may apply an object identification algorithm such as a face recognition (eg, face recognition; FR) algorithm. In addition, a detection algorithm and an object identification algorithm (eg, human detection; HD) may be separately applied to the image according to the location of the object. For example, the first image processing unit 111a applies an object detection algorithm to a captured image of an object in the first area, and the second image processing unit 111b applies an object detection algorithm to a captured image of an object in the second area. Identification algorithms can be applied.

Referring to FIG. 6A, the doorbell device 10 includes a camera 112 and a motion detector 116, and detects the motion of an object approaching the camera 112 using the motion detector 116, Distance information between the object and the camera may be obtained. The motion detector 116 may include an infrared rays sensor. It is possible to determine whether an object is in the first area or the second area through the motion detector 116 .

When the motion detector 116 determines that the object is in the first area, a first signal is transmitted to the camera 112 so that the camera 112 can capture an image of the object in the first area. Similarly, when the motion detector 116 determines that the object is in the second area, a second signal is transmitted to the camera 112 so that the camera 112 can capture an image of the object in the first area.

In addition, when the motion detector 116 determines that the object is in the first area, a first signal may be transmitted to the processor 100 and the first image processing unit 111a may operate in response thereto. Similarly, when the motion detector 116 determines that the object is in the second area, a second signal may be transmitted to the processor 100 and the second image processing unit 111b may operate in response thereto.

The first image processing unit 111a may detect an object from an image captured by the camera 112 of the first region. The first image processing unit 111a may detect an object within the first area by applying an object detection algorithm to the image.

The first image processing unit 111a may detect an object from an image using a motion detection (MD) or human detection (HD) algorithm. Since the human detection algorithm detects the shape of a person, reliable event detection is possible compared to motion detection algorithms that detect changes in the surrounding environment, such as changes in illumination and changes in tree branches caused by wind, as well as human motion.

When the detected object is within the second area, the second image processing unit 111b may identify the object from an image captured by the camera 112 of the second area. The second image processing unit 111b may identify an object entering the second area by applying an object identification algorithm to the image. The second image processing unit 111b may identify an object using a face recognition (FR) algorithm.

For example, the second image processing unit 111b may distinguish whether the detected object is a registrant or an unregistered person by comparing a previously registered face pattern with a face pattern of a detected object.

As another embodiment, referring to FIG. 6B , the image processing unit 111 divides the image 200 acquired by the camera into at least two regions 210 and 220, and an object detection algorithm and an object identification algorithm for each region. can be applied separately. For example, the first image processing unit 111a applies an object detection algorithm to the acquired image portion 220 corresponding to the first area, and the second image processing unit 111b applies the acquired image portion 220 corresponding to the second area. An object identification algorithm may be applied to the image portion 210 .

In other words, a visitor may be detected and/or identified by applying a different image processing algorithm for each region 210 or 220 in the image 200 acquired by the camera. For example, the central portion of the image 200 may be region A 210 corresponding to the second region, and other portions may be region B 220 corresponding to the first region.

Therefore, as shown in FIG. 6B, if an object exists in area B 220 of the acquired image, the first image processing unit 111a drives an object detection algorithm for detecting the object, and If an object exists in area A 210, the second image processing unit 111b may drive an object identification algorithm to identify the object.

In the case of the embodiment of the present invention, the object identification algorithm can identify the face 212 of the object of the area A 210 and also the company mark information 214 . In the case of the embodiment of the present invention, when recognizing the object, if the company mark information 214 is also identified in addition to the face 212 of the object, the object is stored as a separate group, for example, a courier company group, to provide more accurate information to the user. can provide information.

As another embodiment, the object detection algorithm of the first image processing unit 111a and the object identification algorithm of the second image processing unit 111b may be simultaneously activated. In this case, if the object is located in the first area, the object identification algorithm cannot operate normally and only the object detection algorithm operates, but if the face of the object can be recognized, that is, the second area When entering the inside, the object identification algorithm operates and the object can be identified.

Also, the image processing unit 111 may store in the storage unit 105 images of objects in the first area and objects in the second area, respectively, in the storage unit 105, and the image processing unit 111 determines the time when the object entered each area. It can be matched with the image and saved.

The sound processing unit 113 may include an audio codec and process sound received through the microphone 114 . The sound processing unit 113 may detect an acoustic event set by the user, such as a glass breaking sound, an alarm sound, a collision sound, a shout, a scream, a cry, or a sound exceeding a threshold value, from the received sound. there is. The sound processing unit 113 may store an image captured when an acoustic event is detected in the storage unit 105 . The sound processing unit 113 may match and store the sound event detection time and sound to the image.

The alarm unit 115 may select an alarm sound when an image event is detected by the image processing unit 111, for example, when an object is identified. The alarm sound may be set for each registrant, such as, for example, a first alarm sound corresponding to a first registrant and a second alarm sound corresponding to a second registrant. The registrants may include family members, acquaintances, existing visitors, and the like. The user may define a name for each registrant and may set a different alarm sound for each registrant.

The alarm unit 115 may select an alarm sound corresponding to the identified object. In addition, the user may set a third alarm sound corresponding to non-registered users. In the embodiment of the present invention, by setting different alarm sounds for each registrant, the user can individually determine whether the visitor is a family member, a previous visitor, a new visitor, etc. according to the alarm sound before the visitor presses the bell. .

Accordingly, it is possible to quickly follow up on the visitor. The alarm unit 115 may select an alarm sound when a sound event is detected by the sound processor 113 . The alarm sound may be set differently for each type of sound event.

The alarm sound selected by the alarm unit 115 may be output to the outside through a speaker. In addition, the alarm unit 115 may transmit information on the selected alarm sound to the remote user terminal 30 through the communication unit 103 . The user terminal 30 may output a corresponding alarm sound source to the user based on the alarm sound information.

Upon detecting a video event, the alarm unit 115 may transmit at least one of video event information (corresponding video, identified object or visitor information) and alarm sound information to the user terminal 30 . When an acoustic event is detected, the alarm unit 115 may transmit at least one of acoustic event information (corresponding video and audio) and alarm sound information to the user terminal 30 .

7 is a diagram for explaining an example of video event detection according to an embodiment of the present invention.

5 to 7, the camera 112 of the doorbell device 10 captures the object OB entering the first area of the first distance R1 at the first time t1. image can be obtained.

When the image processing unit 111 detects an object OB from the first image using an object detection algorithm, the alarm unit 115 may transmit a primary motion event push to the user terminal 30 through the communication unit 103. there is. The alarm unit 115 may transmit the first image to the user terminal 30 together with an event push.

The camera 112 of the doorbell device 10 may acquire a second image of the object OB entering the second area of the second distance R2 at the second time t2. The second distance R2 is smaller than the first distance R1.

When the image processing unit 111 identifies an object OB from the second image using an object identification algorithm, the alarm unit 115 may transmit a secondary motion event push to the user terminal 30 through the communication unit 103. there is. The alarm unit 115 may transmit the second image to the user terminal 30 together with an event push.

The alarm unit 115 may transmit at least one of identified object (OB) information and alarm sound information set therein to the user terminal 30 together with the second image. The identified object (OB) information may be video or text.

The user terminal 30 may select and output a corresponding alarm sound based on the identified object (OB) information and/or alarm sound information. In one embodiment, the user terminal 30 determines the identified object (OB) received from the doorbell device 10 based on the alarm tone relationship (table) for each registrant stored in the user terminal 30 and the plurality of alarm tones. An alarm sound corresponding to information (ie, visitor information) may be selected and output. For example, the alarm unit 115 may transmit information indicating that the identified object OB is a first registrant to the user terminal 30 .

The user terminal 30 may select and output the alarm sound set for the first registrant from the stored alarm sound relationship for each registrant.

In another embodiment, the user terminal 30 may store a plurality of alarm tones and select and output an alarm tone corresponding to the alarm tone information received from the doorbell device 10 . For example, the alarm unit 115 may transmit information indicating that the identified object OB is a first registrant and information indicating a first alarm sound to the user terminal 30 .

The user terminal 30 may output a pre-stored first alarm sound. In an embodiment of the present invention, the doorbell device 10 may transmit visitor information to a remote user before the visitor presses the bell 109 of the doorbell device 10.

The user can obtain reliable event information and visitor detailed information in advance with reduced false alarms. Thus, users can follow up on unwanted visitors or intruders. Also, the user can take necessary action on the intended visitor.

The doorbell device 10 may transmit a bell event push to the user terminal 30 through the communication unit 103 when a visitor presses the bell 109 of the doorbell device 10 . The user can select a video mode or an audio mode in the user terminal 30 . In the video mode, the user can make a real-time video call with the visitor. In voice mode, users can make real-time voice calls with visitors.

In another embodiment, the camera 112 of the doorbell device 10 is the object OB1 entering the first area of the first distance R1 at the third time t3 and the second distance R2. A third image obtained by photographing the object OB2 entering the second area may be acquired.

The first image processing unit 111a applies an object detection algorithm to the third image to detect an object OB1 entering the first area, and the second image processing unit 111b applies an object identification algorithm to detect the object OB1. An object OB2 entering the area can be identified. Accordingly, the image processing unit 111 may store a partial image of the object OB1 entering the first area and a partial image of the object OB2 entering the second area. Also, the alarm unit 115 may transmit a partial image of the object OB1 entering the first area and a partial image of the object OB2 entering the second area to the user terminal 30 .

8 is a diagram schematically illustrating a security system according to another embodiment of the present invention, and FIG. 9 is a flowchart schematically illustrating the operation of the security system shown in FIG. 8 .

Referring to FIGS. 8 and 9 , the doorbell device 10 may communicate with the user terminal 30 and/or the server 50 through a network.

The embodiment of FIG. 8 is different from the embodiment of FIG. 1 in that a server 50 is added. Hereinafter, a description will be given focusing on differences, and detailed descriptions of overlapping contents of the same configuration will be omitted.

The server 50 may share and store information previously stored in the storage of the doorbell device 10 . In addition, the server 50 may include an artificial intelligence device such as Alexa, and the user terminal may receive event information from the artificial intelligence device.

The doorbell device 10 may detect a motion event from an image acquired by a camera (S11).

The doorbell device 10 may detect a primary motion event for detecting an object in the first area and a secondary motion event for identifying an object in the second area. The doorbell device 10 may detect a sound event from sound acquired by a microphone (S13).

When an event is detected, the doorbell device 10 may transmit an event push to the user terminal 30 (S12 and S14). The doorbell device 10 may transmit information on an alarm sound corresponding to a corresponding video to the user terminal 30 together with an event push.

The user terminal 30 receives the event push, and the user can grasp the surrounding situation of the doorbell device 10 . When the doorbell device 10 detects a bell pressing event (S15), it may transmit a bell event push to the user terminal 30 (S16).

The user may select a video mode and/or a sound mode on the user terminal 30 to conduct a video call and/or an audio call with a visitor in real time. When a motion event is detected, the doorbell device 10 may transmit motion event information to the server 50 (S51).

Motion event information may include an event video and an event occurrence time. When a sound event is detected, the doorbell device 10 may transmit sound event information to the server 50 (S52). The sound event information may include a sound detected as an event, an image captured when a sound event is detected, and an event occurrence time.

The server 50 may generate situation information by collecting and analyzing event information (S53). The server 50 may transmit the situation information to the user terminal 30 (S54). Context information may be provided in the form of a message (text) suitable for a situation composed of words, phrases, sentences, and the like. The user terminal 30 may request event information corresponding to the received situation information from the server 50 (S55).

The server 50 may provide event information to the user terminal 30 in response to a request for event information from the user terminal 30 (S56). In the above-described embodiment, the doorbell device 10 transmits an alarm sound source to the user terminal 30, but in another embodiment, an alarm sound corresponding to a registrant is set in the server 50, and the server 50 ) may select an alarm sound based on event information received from the doorbell device 10 and transmit the corresponding sound source to the user terminal 30 . Alternatively, the server 50 may transmit alarm sound information to the user terminal 30, and the user terminal 30 may select and output a sound source based on the alarm sound information.

The server 50 may classify the event information in units of a specific period, summarize the event information in units of a predetermined period, for example, in units of days or weeks, and transmit the information to the user terminal.

10 and 11 are flowcharts illustrating the operation of a security system according to another embodiment of the present invention.

Referring to FIGS. 3, 6, and 10 , first, the camera 112 of the doorbell device 10 captures a first image of the object OB within the first area (S100). At this time, it is possible to determine whether the object is in the first area or the second area through the motion detector 116 of the doorbell device. The motion detector 116 may transmit a first signal to the processor 100 when it is determined that the object is in the first area.

When the first signal is received, the first image processing unit 111a processes the image by applying an object detection algorithm to the first image obtained by capturing the object in the first area, and detects the object in the first image through this ( S110).

Then, the camera 112 captures a second image of an object within the second area (S120). The motion detector 116 may transmit a second signal to the processor 100 when it is determined that the object is in the second area.

When the second signal is received, the second image processing unit 111b performs image processing by applying an object identification algorithm to a second image obtained by capturing an object in the second area, and identifies an object in the first image ( S130).

Thereafter, the second image including the identified object and event information corresponding to the second image are stored in the storage 105 and/or the server 50 of the doorbell device. (S140)

Then, the server 50 classifies the event information in units of a specific period, summarizes the event information in units of a predetermined period, for example, in units of days or weeks, and transmits the summary to the user terminal. (S150)

As another embodiment, referring to FIGS. 3, 4, 6, and 11 , the camera 112 captures a first image of the object OB within the first area (S200). The motion detector 116 may transmit a first signal to the processor 100 when it is determined that the object is in the first area.

When the first signal is received, the first image processing unit 111a processes the image by applying an object detection algorithm to the first image obtained by capturing the object in the first area, and detects the object in the first image through this ( S210).

Then, the camera 112 captures a second image of an object within the second area (S220). The motion detector 116 may transmit a second signal to the processor 100 when it is determined that the object is in the second area.

When the second signal is received, the second image processing unit 111b performs image processing by applying an object identification algorithm to a second image obtained by capturing an object in the second area, and identifies an object in the first image ( S230).

Thereafter, the second image including the identified object and event information corresponding to the second image are stored in the storage 105 and/or the server 50 of the doorbell device. (S240)

As described above with reference to FIG. 4 , the user may select video events in which a specific object is photographed through the screen 300 of the terminal and store them in the doorbell database 105b and/or the server. That is, the user may access the storage unit 105 and/or the server through the user terminal 30 and classify specific video events into separate categories and store them in the database 105b.

Accordingly, the server 50 may compare the second image including the identified object with images stored in the database (S250), and if the identified object matches the image stored in the database, the matching information Event information including may be provided to the user terminal 30 . (S260)

As described above, the present invention has been described by specific details such as specific components and limited embodiments and drawings, but these are provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , Various modifications and variations from these descriptions are possible to those skilled in the art in the field to which the present invention belongs.

Therefore, the spirit of the present invention should not be limited to the described embodiments, and it will be said that not only the claims to be described later, but also all modifications equivalent or equivalent to these claims belong to the scope of the present invention. .

Claims (20)

a doorbell detecting an object in the first area and identifying an object in the second area;
a user terminal receiving information corresponding to the identified object from the doorbell; and
A communication network providing one or more communication channels between the user terminal and the doorbell,
The doorbell,
A camera that acquires an image of an object, the camera,
obtaining a first image of an object entering a first area at a first distance from the camera and obtaining a second image of an object entering the second area at a second distance from the camera;
The first area is a range of a distance at which motion of the object can be detected in an image captured by the camera,
The second area is a range of distances at which the object can be identified in the image captured by the camera,
the second distance from the camera is shorter than the first distance;
a first image processing unit for detecting the object without identifying the object in the first image by applying an object detection algorithm; and the detected object in the first image by applying an object identification algorithm different from the object detection algorithm. an image processing unit including a second image processing unit for identifying the same object in a second image;
a storage unit for storing the second image corresponding to the identified object; and
A security system comprising an alarm unit for activating an alarm signal according to whether the identified object corresponds to a predetermined object. According to claim 1,
Further comprising a server that collects and analyzes event information to generate status information and transmits the status information to the user terminal;
The security system, characterized in that the status information is provided in a message format suitable for the situation in which the object is detected and / or identified. According to claim 2,
The security system according to claim 1 , wherein the server classifies the event information into predetermined periods, summarizes the event information, and transmits the summary to the user terminal. According to claim 1,
The security system of claim 1 , wherein the user terminal accesses the storage unit, selects a specific event of the storage unit, and stores the specific event in separate categories. According to claim 1,
The doorbell,
When the object is located in the first area, a first signal is transmitted to the camera to capture the first image, and when the object is located in the second area, a second signal is transmitted to capture the second image. Security system characterized in that it further comprises a motion detector for transmitting to the camera. According to claim 1,
The doorbell,
If the object is located in the first area, a first signal is transmitted to the first image processing unit to detect the object in the first image, and if the object is located in the first area, an object in the second image is transmitted. The security system further comprises a motion detector for transmitting a second signal to the second image processing unit for identification. delete A camera that acquires an image of an object, the camera,
obtaining a first image of an object entering a first area at a first distance from the camera and obtaining a second image of an object entering a second area at a second distance from the camera;
The first area is a range of a distance at which motion of the object can be detected in an image captured by the camera,
The second area is a range of distances at which the object can be identified in the image captured by the camera,
the second distance from the camera is shorter than the first distance;
a first image processing unit for detecting the object without identifying the object in the first image by applying an object detection algorithm; and the detected object in the first image by applying an object identification algorithm different from the object detection algorithm. an image processing unit including a second image processing unit for identifying the same object in a second image;
a storage unit for storing the second image corresponding to the identified object; and
A doorbell device comprising an alarm unit for activating an alarm signal according to whether the identified object corresponds to a predetermined object. According to claim 8,
A doorbell device further comprising a motion detector for determining whether the object is in the first area or the second area. According to claim 9,
The motion detector transmits a first signal to the camera to capture the first image when the object is located in the first area, and to capture the second image when the object is located in the second area. Doorbell device, characterized in that for transmitting a signal to the camera. According to claim 9,
The motion detector transmits a first signal to the first image processing unit to detect an object in the first image when the object is located in the first area, and transmits a first signal to the second image processing unit when the object is located in the first area. A doorbell device characterized in that it transmits a second signal to the second image processing unit to identify an object in the image. According to claim 8,
The doorbell device, characterized in that the image processing unit identifies all objects in the second image. delete According to claim 8,
The image processing unit divides the image acquired by the camera into first and second parts, and applies different algorithms to the image data of the first and second parts. According to claim 14,
The first image processing unit applies an object detection algorithm to a first portion corresponding to the first area, and the second image processing unit applies an object identification algorithm to a second portion corresponding to the second area. Characterized by a doorbell device. According to claim 15,
The second part is a doorbell device, characterized in that the central part of the image acquired by the camera. According to claim 8,
Microphone installed outside the door;
The doorbell device further comprises a sound processing unit that detects a sound event received from the microphone and stores an image captured by the camera in the storage unit when the sound event is detected. capturing a first image of an object entering a first area at a first distance from the camera;
The first area is a range of a distance at which motion of the object can be detected from an image captured by the camera;
detecting the object in the first image without identifying the object by applying an object detection algorithm;
capturing a second image of an object entering a second area at a second distance from the camera;
The second area is a range of distances at which the object can be identified in the image captured by the camera,
the second distance from the camera is shorter than the first distance;
identifying the same object in the second image detected in the first image by applying an object identification algorithm different from the object detection algorithm;
storing the second image corresponding to the identified object; and
A method of operating a security system comprising transmitting information corresponding to the identified object to a user terminal. According to claim 18,
The security system operating method of claim 1, further comprising classifying the information into predetermined periods and summarizing the information for each predetermined period and transmitting the information to the user terminal. According to claim 18,
comparing the second image including the identified object with pre-stored images; and
and transmitting information corresponding to the same identified object as the pre-stored images to the user terminal.

Images