KR102628414B1 - Image capturing system and network system supporting privacy mode - Google Patents

Abstract

An image capturing system according to an embodiment of the present invention includes a storage medium configured to store a plurality of image information sets respectively corresponding to a plurality of reference images, and operations related to capturing an image and the captured image in a privacy mode. It includes an image processing device that is selectively controlled according to the selected privacy level. The image processing device is configured to detect a face image from a captured image and select one of a plurality of privacy levels according to an image information set that matches the detected face image among the plurality of image information sets.

Description

Video capture system and network system supporting privacy mode {IMAGE CAPTURING SYSTEM AND NETWORK SYSTEM SUPPORTING PRIVACY MODE}

The present invention relates to an image capture system and network system supporting privacy mode.

Recently, video monitoring systems have been spreading at a rapid pace for various purposes such as public security and safety. These video surveillance systems can provide video data to other devices through a network, thus enabling a remote device to play back captured images or allow a remote device to store captured images.

As the use of these video surveillance systems increases, protection of individual privacy becomes an issue. Since video surveillance systems support various functions related to captured video, users may wish to disable video detection functions or use only some of the video detection functions as needed. In this case, the user can activate the privacy mode by setting the video surveillance system and thereby disable some functions supported by the video surveillance systems. However, this method causes inconvenience in that the user must manually set the video surveillance system to activate the privacy mode, and may not effectively protect individual privacy.

The content described above is only intended to help understand the background technology of the technical ideas of the present invention, and therefore, it cannot be understood as content corresponding to prior art known to those skilled in the art of the present invention.

The video capture system and network system according to an embodiment of the present invention can automatically operate in privacy mode to provide improved user convenience.

An image capturing system according to an embodiment of the present invention includes a storage medium configured to store a plurality of image information sets each corresponding to a plurality of reference images; and an image processing device that, in a privacy mode, selectively controls capture of an image and operations related to the captured image according to a selected privacy level, wherein the image processing device detects a face image from the captured image, and select one of a plurality of privacy levels according to an image information set that matches the detected face image among the plurality of image information sets.

The image processing device is configured to acquire facial images according to shooting of the image, generate the plurality of image information sets in association with the acquired facial images, and store the plurality of image information sets in the storage medium. It can be configured.

The image processing device determines the privacy level corresponding to the image information set according to the number of face images associated with each image information set among the acquired facial images, thereby determining the privacy levels corresponding to the plurality of image information sets. and may be configured to store the determined privacy levels in the storage medium.

When the image processing device receives a user input indicating a privacy level corresponding to a selected image information set among the plurality of image information sets, the user inputs the privacy level stored in association with the selected image information set. It can be configured to be modified according to.

The detected face image includes first and second face images, the first and second face images match first and second image information sets of the plurality of image information sets, respectively, and the first The first and second image information sets correspond to first and second privacy levels, respectively, of the plurality of privacy levels, and the image processing device selects the one with the highest priority among the first and second privacy levels. and the priorities of the first and second privacy levels may be determined in advance.

The image capturing system further includes a communicator, wherein the image processing device sends event information through the communicator when there is no image information set that matches the detected face image among the plurality of image information sets. Can be configured to transmit to an external device.

The image processing device operates in the privacy mode when there is a set of image information that matches the detected face image among the plurality of image information sets, and returns to the privacy mode a predetermined time after starting to operate in the privacy mode. Can be configured to terminate.

The image processing device controls at least some of the operations when a first privacy level is selected among the plurality of privacy levels, and controls at least some of the operations when a second privacy level is selected among the plurality of privacy levels. It can be configured to control some other part.

The image capture device may stop capturing the image when a first privacy level is selected among the plurality of privacy levels.

The image capture system includes a non-volatile storage medium; and a communicator, wherein the operations include transmitting the captured image to an external device through the communicator, and the image processing device is configured to transmit the captured image to an external device when a second privacy level is selected among the plurality of privacy levels. The operation of transmitting the captured image to the external device can be stopped.

The operations include at least one of a first operation of storing the captured image in the non-volatile storage medium and a second operation of storing the captured image in an external storage medium through the communicator, and the image processing device The first operation and the second operation may be stopped when a third privacy level is selected among the plurality of privacy levels.

The image capturing system may further include a communicator that provides communication with an external device, and the image processing device may be configured to notify the external device of the selected privacy level through the communicator.

Another aspect of the present invention relates to a network system. The network system includes a plurality of image capture devices connected to a network, wherein a first image capture device among the plurality of image capture devices includes a communicator that provides communication through the network; a storage medium configured to store a plurality of image information sets each corresponding to a plurality of reference images; and an image processing device that, in a privacy mode, selectively controls capture of an image and operations related to the captured image according to a selected privacy level, wherein the image processing device detects a face image from the captured image, and select one of a plurality of privacy levels according to an image information set that matches the detected face image among the plurality of image information sets. At least one second image capture device among the plurality of image capture devices enters the privacy mode when one of the plurality of privacy levels is selected in the first image capture device.

The network system further includes a computer device that communicates with the plurality of image capture devices through the network, wherein the first image capture device notifies the selected privacy level to the computer device through the communicator, and the computer device may control the second image capture device to operate in the privacy mode according to the selected privacy level.

In the privacy mode, the second image capture device can selectively control the operations in the same manner as the first image capture device.

According to an embodiment of the present invention, the video recording system and the network system can automatically operate in privacy mode, providing improved user convenience.

Figure 1 is a block diagram showing a network system including a camera.
Figure 2 is a block diagram showing an image capturing system according to an embodiment of the present invention.
Figure 3 is a flowchart showing a method of providing a privacy mode based on a captured image according to an embodiment of the present invention.
Figure 4 is a flowchart showing a method of providing a privacy mode based on a captured image according to another embodiment of the present invention.
Figure 5 is a flowchart showing an example of a method for returning to normal mode after entering privacy mode.
6 is a flowchart showing an embodiment of a method for providing reference image information sets.
Figure 7 is a flowchart showing an embodiment of a method for determining a privacy level corresponding to a set of reference image information.
Figure 8 is a table showing privacy levels determined to correspond to reference image information sets.
Figure 9 is a flowchart showing another embodiment of a method for determining a privacy level corresponding to a set of reference image information.
FIG. 10 is a block diagram showing an example computer system for implementing the imaging system of FIG. 2.
Figure 11 is a block diagram showing a network system including a plurality of cameras.
FIG. 12 is a flowchart showing operations of the first to k cameras of FIG. 11 and the computer device.

Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the attached drawings. It should be noted that in the following description, only the parts necessary to understand the operation according to the present invention will be described, and the description of other parts will be omitted in order to not obscure the gist of the present invention. Additionally, the present invention is not limited to the embodiments described herein and may be embodied in other forms. However, the embodiments described herein are provided to explain in detail enough to enable those skilled in the art to easily implement the technical idea of the present invention.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected," but also the case where it is "indirectly connected" with another element in between. . The terminology used herein is for the purpose of describing specific embodiments and is not intended to limit the invention. Throughout the specification, when a part is said to “include” a certain element, this means that it may further include other elements rather than excluding other elements, unless specifically stated to the contrary. “at least one of X, Y, and Z”, and “at least one selected from the group consisting of It can be interpreted as any combination (e.g., XYZ, XYY, YZ, ZZ). As used herein, “and/or” includes any combination of one or more of the constituents.

Figure 1 is a block diagram showing a network system including a camera.

Referring to FIG. 1, the network system 100 includes a network (105), a camera (110), an event management server (120), a cloud database (130), and a user terminal ( 140, User Terminal). Each of the camera 110, event management server 120, cloud database 130, and user terminal 140 is one or more devices for executing commands such as computer programs or data stored in a computer-readable medium. The inclusion of processors, memories, and other suitable components may enable the various applications and steps described herein. For example, computer programs and/or instructions may be stored on a medium readable by a computer internal or external to the device.

The network 105 connects the camera 110, the event management server 120, the cloud database 130, and the user terminal 140. Network 105 may include at least one of a public network, at least one private network, a wired network, a wireless network, another suitable type of network, and combinations thereof. Each of the camera 110, the event management server 120, the cloud database 130, and the user terminal 140 may include at least one of a wired communication function and a wireless communication function, and thus may be connected via the network 105. They can communicate with each other.

The camera 110 communicates with at least one of the event management server 120, the cloud database 130, and the user terminal 140 through the network 105. The camera 110 includes, for example, an image sensor that converts optical signals input through a lens into electronic signals, and a microphone that converts voice into electronic signals, and can capture images using these. Video recording is turned on/off according to various methods and is performed when the camera 110 is in an on state.

The camera 110 processes captured images (or multimedia) according to appropriate methods. The camera 110 stores image data (or multimedia data) based on the captured image in an internal non-volatile storage medium (115), uploads it to the cloud database 130, or uploads it to the user terminal 140. ), for example, can be transmitted through real-time communication.

The camera 110 may generate event information based on the captured image. For example, various event types are defined, such as motion detection, voice detection, face detection, body detection, fire detection, crying sound detection, gun sound detection, and impact sound detection, and the camera 110 detects event types from captured images. When at least one of the following is detected, corresponding event information can be generated. Camera 110 may include components for detecting these event types. The camera 110 may transmit the generated event information to the event management server 120.

The event management server 120 may process the transmitted event information in various ways and provide event information or processed information when the user terminal 140 accesses the event management server 120.

The cloud database 130 may store image data provided from the camera 110 and provide the image data to the user terminal 140 in response to a request from the user terminal 140 . The camera 110 may provide image data stored in the non-volatile storage medium 115 in response to a request from the user terminal 140. When the non-volatile storage medium 115 is provided in a detachable manner from the camera 110, the user can play image data by connecting the non-volatile storage medium 115 to various devices, including the user terminal 140. .

The user terminal 140 may be a personal computer (PC), a desktop computer, a laptop computer, a portable computer, a personal digital assistant (PDA), a smart phone, a tablet PC, or a video device. It may include at least one device, such as a gaming device, that can communicate with the camera 110, the event management server 120, and the cloud database 130 through the network 105 to perform information processing and playback of image data. there is.

Figure 2 is a block diagram showing an image capturing system according to an embodiment of the present invention.

Referring to FIG. 2, the image capture system 200 includes a lens 205, an image processing device 210, a non-volatile storage medium 220, and a memory 230, and Includes.

The image processing device 210 may capture an image, process the captured image, and perform various operations related to the processed image. That is, the image processing device 210 is configured to perform operations related to capturing an image and operations related to the captured image.

The image processing device 210 may operate in privacy mode. The image processing device 210 may detect a face image from a captured image and enter the privacy mode based on the detected face image. During the privacy mode, the image processing device 210 may stop at least some of the image capturing and operations related to the captured image performed in the normal mode. During the privacy mode, the image processing device 210 may perform additional operations related to the captured image, for example, distorting the image.

The image processing device 210 provides a plurality of privacy levels in privacy mode. The image processing device 210 may select one of a plurality of privacy levels based on the detected face image. Capture of video and operations associated with the captured video are selectively stopped and/or controlled depending on the selected privacy level.

The image processing device 210 includes an image sensor (211), an image processor (212), a storage interface (213, Storage I/F), a network interface (214, Network I/F), and a main processor ( 215, Main Processor), a face detector (216, Face Detector), and a privacy level controller (217, Privacy Level Controller).

The image sensor 211 is connected to the lens 205. The image sensor 211 may operate in response to control from the main processor 215. The image sensor 211 is configured to convert the optical signal received through the lens 205 into an electrical signal and digitize the converted electrical signal to generate an original image (RWIMG). For example, the image sensor 211 may include an analog-to-digital converter.

The image processor 212 is connected to the image sensor 211. The image processor 212 may operate in response to control from the main processor 215. The image processor 212 is configured to receive a raw image (RWIMG) and process the raw image (RWIMG) to generate a processed image (PRIMG). For example, the image processor 212 may adjust the size of the original image (RWIMG), adjust the resolution of the original image (RWIMG), or perform various algorithms for other necessary processing.

The storage interface 213 interfaces between the non-volatile storage medium 220 and the image processing device 210. The storage interface 213 may operate in response to control of the main processor 215. The storage interface 213 may receive the processed image PRIMG and write it to the non-volatile storage medium 220 .

The network interface 214 interfaces between the communicator 240 (or transceiver) and the image processing device 210. The network interface 214 may operate in response to control of the main processor 215. The network interface 214 receives the processed image (PRIMG) and transmits it via the network 105 (see FIG. 1) to a cloud database 130 (see FIG. 1) and/or a user terminal 140 (see FIG. 1). The communicator 240 can be controlled to transmit to an external device. The communicator 240 may transmit and receive signals between the image capturing system 200 and other devices through the network 105 (see FIG. 1).

The main processor 215 controls overall operations of the image processing system 200. The main processor 215 may control the image sensor 211, the image processor 212, the storage interface 213, the network interface 214, the face detector 216, and the privacy level controller 217. In embodiments, the main processor 215 may control overall operations of the image processing system 200 by executing software and/or firmware. Such software and/or firmware may be stored in memory inside or outside the main processor 215 and executed by the main processor 215.

The main processor 215 controls the transition between the normal mode and privacy mode of the image processing device 210. Furthermore, the main processor 215 performs operations performed by the image sensor 211 and the main processor 215, for example, associated with the shooting of an image, and operations associated with the captured image, for example, according to the selected privacy level. Operations performed by the image processor 212, the storage interface 213, the network interface 214, and the main processor 215 may be controlled to be selectively performed.

Face detector 216 is coupled to image processor 212. Face detector 216 may operate in response to control of main processor 215. Face detector 216 receives a processed image (PRIMG) provided by image processor 212. The face detector 216 is configured to detect a face included in the processed image (PRIMG) and provide face image information (FI) according to the detection result. The face detector 216 detects an area containing a face image among the processed images (PRIMG), acquires and/or stores the detected face image, and stores feature information, such as feature points, of the detected face image in the face image. It can be provided as information (FI).

Privacy level controller 217 is coupled to face detector 216. Privacy level controller 217 may operate in response to control of main processor 215. The privacy level controller 217 receives face image information (FI) from the face detector 216, and receives the first to nth reference image information sets (RFI1 to RFIn) from the memory 230 (n is 2 an integer greater than or equal to). The first to nth reference image information sets (RFI1 to RFIn) stored in the memory 230 may respectively correspond to the first to nth reference images. For example, the first to nth reference images may be selected by the user, and the first to nth reference image information sets (RFI1 to RFIn) may be obtained from the first to nth reference images. Each reference image information set may include feature information of the corresponding reference image. Each set of reference image information may be generated by face detector 216 and stored in memory 230.

The memory 230 may be in the form of a memory device or system known or developed in the art, and is configured to store first to nth reference image information sets (RFI1 to RFIn). Additionally, the memory 230 may further store privacy levels associated with each of the first to nth reference image information sets (RFI1 to RFIn). For example, memory 230 may store privacy level information indicating which privacy level corresponds to each reference image information set. The first to nth reference image information sets (RFI1 to RFIn) and their privacy levels may be stored in the non-volatile storage medium 220 and loaded into the memory 230 in response to control of the main processor 215. You can. The first to nth reference image information sets (RFI1 to RFIn) loaded in the memory 230 and their privacy levels may be provided to the privacy level controller 217.

The privacy level controller 217 is configured to compare the face image information (FI) with the first to nth reference image information sets (RFI1 to RFIn) and report the comparison result to the main processor 215. The privacy level controller 217 may enable the alarm signal ALR when there is reference image information that matches the face image information FI among the first to nth reference image information sets RFI1 to RFIn. The privacy level controller 217 may disable the alarm signal ALR when there is no reference image information matching the facial image information FI. The processor 215 responds to the disabled alarm signal ALR. Thus, event information can be generated and the generated event information can be transmitted to the event management server 120 through the network interface 214. For example, when a stranger is photographed, reference image information matching face image information (FI) may not exist, and in this case, event information may be generated.

The privacy level controller 217 selects one of the first to nth privacy levels according to the reference image information set that matches the face image information (FI) among the first to nth reference image information sets (RFI1 to RFIn). And, the selected privacy level (PLSEL) can be notified to the main processor 215. That is, the privacy level controller 217 selects a privacy level corresponding to the matching set of reference image information.

In embodiments, the privacy level corresponding to each reference image information set may be determined by user input when registering the reference image. For example, the privacy level corresponding to the first user may be the highest, the privacy level corresponding to the second user may be the lowest, and the privacy level corresponding to the third user may be in the middle.

The processor 215 may control the image processing device 210 or the image capturing system 200 according to the selected privacy level. That is, operations performed by the image processing device 210 or the image capturing system 200 change depending on the selected privacy level.

For example, when the first privacy level is selected, the main processor 215 may control the image sensor 211 to stop capturing images. As filming of video is stopped, video streaming and video recording may not be performed. For example, all video capture functions provided by surveillance cameras may be interrupted. The main processor 215 may control components of the image capture system 200 so that the user can visually notice that image capture is not being performed. For example, in the case of the image capture system 200 that supports a function of moving the lens 205, such as pan tilt, the main processor 215 allows the user to capture images when the first privacy level is selected. The lens 205 can be controlled to point in a different direction so that it can be seen visually that it is not being performed. For example, in the case of the image capture system 200 including a light emitting device indicating capture of an image, the main processor 215 may visually inform the user that image capture is not performed when the first privacy level is selected. The above light emitting device can be controlled to stop emitting light or to emit light of a different color. Various event types are defined, such as motion detection, voice detection, face detection, body detection, fire detection, crying sound detection, gun sound detection, and impact sound detection, and the main processor 215 detects at least one of the event types from the captured image. When detected, event information may be generated, and the generated event information may be stored in the event management server 120 through the network interface 214. When the first privacy level is selected, operations for generating event information may also be stopped.

When the second privacy level is selected, the main processor 215 captures the image and transmits the captured image to an external device such as a cloud database 130 (see FIG. 1) through the network interface 214. can be stopped. In embodiments, for security purposes, the main processor 215 may detect an event and, when the event is detected, store the corresponding image in the non-volatile storage medium 220 through the storage interface 213. At this time, the occurrence of the event may not be notified to the user terminal.

When the third privacy level is selected, the main processor 215 may perform image capture but may stop storing the captured image. For example, the main processor 215 may stop storing the captured image in the non-volatile storage medium 220 and storing it in an external device through the network interface 214. In embodiments where image storage is performed based on detection of an event, detection of the event may also be stopped. At this time, the main processor 215 can provide the captured image to an external device in a streaming manner.

However, embodiments of the present invention are not limited to the above examples. Depending on the implementation example of the image processing device 210 or the image capturing system 200, the operations performed by the image processing device 210 or the image capturing system 200 may be modified in various ways, and the operations are controlled at each privacy level. may be changed.

According to an embodiment of the present invention, the image processing device 210 operates in privacy mode based on the detected face image. Compared to the case where entry into privacy mode is controlled by user input, the image processing device 210 can automatically operate in privacy mode when a face image matching the reference face image is detected, thereby improving user convenience. It can be improved.

According to an embodiment of the present invention, the image processing device 210 selectively controls capturing of images and operations related to the captured images according to the selected privacy level. When registering a desired reference image, the user can select a corresponding privacy level, and operations of the image processing device 210 can be controlled depending on which privacy level is selected. Since various privacy levels are provided, not only does user convenience improve, but also the possibility of leakage of captured images, the limited storage space for storing captured images, and the excessive amount of event information accumulated in the event management server 120, etc. Various issues related to video can be adaptively resolved for each user.

In Figure 2, the face detector 216 and privacy level controller 217 are shown as separate components from the main processor 215. However, embodiments of the present invention are not limited thereto. The face detector 216 and privacy level controller 217 are integrated into the main processor 215 so that the main processor 215 can perform the functions of the face detector 216 and privacy level controller 217.

Face detector 216 and privacy level controller 217 may be implemented through software, hardware, firmware, and combinations thereof. For example, face detector 216 and privacy level controller 217 are implemented by software modules that, when executed by main processor 215, provide the functions of face detector 216 and privacy level controller 217. It can be. In this case, the main processor 215 can provide the functions of the face detector 216 and the privacy level controller 217 by loading the corresponding software modules into the working memory and executing the loaded software modules.

Figure 3 is a flowchart showing a method of providing a privacy mode based on a captured image according to an embodiment of the present invention.

Referring to FIG. 3, in step S110, an image captured in normal mode is received. In step S120, a face image is detected from the captured image. The captured image is processed according to predetermined methods, and an area containing a face image can be detected from the processed image. Face images are analyzed by various methods such as Gabor Filter, Principal Component Analysis (PCA), Fisher Discriminant Analysis (FDA), Independent Component Analysis (ICA), Local Binary Feature (LBP), and Support Vector Machine (SVM). It can be detected based on at least one of:

In step S130, it is determined whether the detected face image matches the first reference image. As an example, it is determined whether feature information (eg, feature points) of the detected face image matches a reference image information set of the first reference image. If so, step S160 is performed. If not, step S140 is performed.

Similar to step S130, the detected face image is also compared with other reference images. In this embodiment, for convenience of explanation, a case in which three reference images exist is exemplarily described. In step S140, it is determined whether the detected face image matches the second reference image. If so, step S170 is performed. If not, step S150 is performed. In step S150, it is determined whether the detected face image matches the third reference image. If so, step S180 is performed. If not, step S190 is performed.

When the detected face image matches any one of the first to third reference images, the image processing device 210 (see FIG. 2) operates in privacy mode. The privacy level is selected depending on which reference image matches the detected face image.

In step S160, the image processing device 210 is controlled to operate at the first privacy level. In step S170, the image processing device 210 is controlled to operate at the second privacy level. In step S180, the image processing device 210 is controlled to operate at the third privacy level.

In step S190, when the detected face image does not match the first to third reference images, event information may be generated. The generated event information is transmitted to the event management server 120 (see FIG. 1), and the user terminal 140 (see FIG. 1) can receive event information by accessing the event management server 120.

Figure 4 is a flowchart showing a method of providing a privacy mode based on a captured image according to another embodiment of the present invention.

Referring to FIG. 4, in step S210, a captured image is received. In step S220, a face image is detected from the captured image. Steps S210 and S220 are described similarly to steps S110 and S120 of FIG. 3, respectively. Hereinafter, overlapping descriptions will be omitted.

In step S230, it is determined whether two or more face images are detected from the captured image. If so, step S240 is performed. If not, step S130 of FIG. 3 may be performed.

In step S240, it is determined whether each of the detected face images matches the reference images. Feature information may be extracted from the detected face image, and the extracted feature information may be compared with previously stored reference image information sets (RFI1 to RFIn, see FIG. 2).

In step S250, step S260 or step S270 is performed, depending on whether two or more face images match the reference images.

In step S260, the privacy level with the highest priority is selected among the privacy levels corresponding to the matched reference images. At this time, priorities of privacy levels can be determined in advance. For example, the first privacy level has the highest priority, the second privacy level has a lower priority than the first privacy level, and the third privacy level has a lower priority than the second privacy level.

In step S270, a privacy level corresponding to the matched reference image is selected.

Figure 5 is a flowchart showing an example of a method for returning to normal mode after entering privacy mode.

Referring to FIGS. 2 and 5 , in step S310, the image processing device 210 (FIG. 2) is controlled to operate in a privacy mode. The main processor 215 may control the image processing device 210 according to the selected privacy level.

In step S320, it is determined whether a predetermined time has elapsed after starting to operate in privacy mode. For example, the image processing system 200 further includes a timer, and whether a predetermined time has elapsed can be determined using the timer.

In step S330, the image processing device 210 is controlled to return to the normal mode. That is, the privacy mode is terminated. For example, the image processing device 210 may operate at the selected privacy level for 30 minutes or 1 hour and then return to the normal mode.

6 is a flowchart showing an embodiment of a method for providing reference image information sets.

Referring to FIGS. 2 and 6 , in step S410, a captured image is received. In step S420, step S430 is performed depending on whether a face image is detected from the captured image. Step S420 may be performed by the video capturing system 200. In this case, detection of the face image may be performed by the face detector 216 of FIG. 2.

As another example, step S420 may be performed by the user terminal 140 (see FIG. 1). In this case, the user terminal 140 may further include software or hardware modules for detecting a face image from the received image.

In step S430, the detected face image may be provided to the user terminal 140. The main processor 215 may transmit the detected facial image to the user terminal 140 through the network interface 214, and the user terminal 140 may display the received facial image.

The user of the user terminal 140 may provide an input to register (or select) the detected face image. At this time, the user may further provide input to determine the corresponding privacy level.

In step S440, when an input for registering a detected face image is received from the user terminal 140, feature information associated with the detected face image is stored as a set of reference image information. An input for registering the detected face image may be received from the user terminal 140 through the communicator 240. The main processor 215 may, in response to an input for registering the detected facial image, generate feature information associated with the detected facial image and store the generated feature information as a set of reference image information in the non-volatile storage medium 220. . The input for registering the detected face image may further include data about the determined privacy level. This determined privacy level may be further stored in non-volatile storage medium 220 in association with a set of reference image information.

Figure 7 is a flowchart showing an embodiment of a method for determining a privacy level corresponding to a set of reference image information.

Referring to FIGS. 2 and 7 , in step S510, a captured image is received. In step S520, step S530 is performed depending on whether a face image is detected from the captured image.

In step S530, it is determined whether there is a reference image information set that matches the detected face image among the pre-stored reference image information sets (RFI1 to RFIn, see FIG. 2). This may be performed by the main processor 215. If not, step S540 is performed. If so, step S550 is performed.

In step S540, a set of reference image information associated with the detected face image is stored. The main processor 215 may generate feature information associated with the detected face image and store the generated feature point information as a set of reference image information in the non-volatile storage medium 220.

In step S550, the count value corresponding to the reference image information set is increased. In step S560, the privacy level corresponding to the reference image information set is determined according to the count value. Steps S550 and S560 may be performed by the main processor 215.

People or objects that are frequently photographed may be the main users of the space where the video capture system 200 is installed or people related thereto. By managing the count value corresponding to each reference image information set and determining the privacy level accordingly, the video capturing system 200 estimates the main user or people related to him/her without user input to set the privacy level of the reference image. , it is possible to determine privacy levels according to their occurrence frequencies and apply those privacy levels when they are photographed. Accordingly, an image capturing system 200 and its operating method that provide improved user convenience can be provided.

Figure 8 is a table showing privacy levels determined to correspond to reference image information sets.

Referring to FIGS. 2 and 8 , the count value and privacy level corresponding to the reference image information set are determined through the embodiment of FIG. 7 . The count value and privacy level may be stored in non-volatile storage medium 220.

In Figure 8, the first reference image information set (RFI1) is shown to have a first count value (CNT1) and thus correspond to a first privacy level (PL1). In addition, the second reference image information set (RFI2) corresponds to the second count value (CNT2) and the third privacy level (PL3), and the nth reference image information set (RFIn) corresponds to the nth count value (CNTn) and the third privacy level (PL3). It is shown to correspond to privacy level 2 (PL2). When the count value is greater than the first threshold, the first privacy level PL1 may be stored in association with the corresponding reference image information set. When the count value is between the first threshold and a second threshold smaller than the first threshold, a second privacy level PL2 may be stored in association with the corresponding reference image information set. When the count value is between the second threshold and a third threshold smaller than the second threshold, the third privacy level PL3 associated with the corresponding reference image information set may be stored.

A custom field indicating whether the privacy level corresponding to each reference image information set has been specified by the user may be further stored. In FIG. 8, the user-specified fields of the first, second, and nth reference image information sets (RFI1, RFI2, RFIn) have privacy levels that can be used by the video capturing system 200 other than the user through the embodiments of FIG. 7. Since it was decided by , it is stored as 'NONE'.

Figure 9 is a flowchart showing another embodiment of a method for determining a privacy level corresponding to a set of reference image information.

Referring to FIGS. 2 and 9 , in step S610, a user input indicating a privacy level corresponding to the reference image information set is received from the user terminal. A user can access the video recording system 200 to view face images corresponding to each set of reference image information, and provide user input to determine the corresponding privacy level through the user terminal. For example, the user may search for a face image corresponding to the second reference image information set (RFI2, see FIG. 8) and set the corresponding privacy level as the first privacy level.

In step S620, the privacy level stored in association with the reference image information set is modified according to user input. For example, the main processor 215 may change the privacy level corresponding to the second reference image information set (RFI2) from the third privacy level (PL3) to the first privacy level (PL1) according to the user input. At this time, the custom field is changed from “NONE” to, for example, “YES”.

FIG. 10 is a block diagram showing an example computer system for implementing the imaging system of FIG. 2.

Referring to FIG. 10, the computer system 1000 includes a bus 1001, a main processor (1002), an internal memory (1003), an image processor (1004), and a storage interface (1005, Storage). I/F), network interface (1006, Network I/F), non-volatile storage medium (1007, Non-volatile Storage Medium), communicator (1008, Communicator), image sensor (1009, Image Sensor), lens (1010, Lens), a motor (1011), a motor controller (1012), and an external memory (1013).

Bus 1001 connects components of computer system 1000 for communication of data, signals, and information. The main processor 1002 may be at least one of a general-purpose processor or a dedicated processor, and may control overall operations of the computer system 1000. The main processor 1002 may perform the functions of the main processor 215 described with reference to FIG. 2 .

The functions of the face detector 216 and the privacy level controller 217 described with reference to FIG. 2 may be performed by the main processor 1002. In this case, the main processor 1002 includes a Face Detection Module, which is a software module that provides the functionality of the face detector 2016 when executed by the main processor 1002, and a Face Detection Module, which when executed by the main processor 1002 The Privacy Level Control Module, which is a software module that provides the functions of the privacy level controller 217, can be loaded into the internal memory 1003 and the loaded modules can be executed.

The internal memory 1003 can store computer programs and instructions to be executed by the main processor 1002. For example, the internal memory 1003 serves as a working memory of the main processor 1002. The internal memory 1003 may be a combination of at least one of random access memory (RAM), read only memory (ROM), and various types of media readable by computers.

The internal memory 1003 may further perform the functions of the memory 230 described with reference to FIG. 2 .

The image processor 1004, storage interface 1005, network interface 1006, non-volatile storage medium 1007, and communicator 1008 are similar to the image processor 212 and storage interface 213 described with reference to FIG. 2. , the network interface 214, the non-volatile storage medium 220, and the communicator 240 are each configured similarly.

The image sensor 1009 and lens 1010 are configured similarly to the image sensor 211 and lens 205 described with reference to FIG. 2 . The motor 1011 is configured to drive the lens 1010, and the motor controller 1012 is configured to control the operation of the motor 1011 in response to control of the semiconductor chip (CHIP).

In embodiments, the main processor 1002, internal memory 1003, and image processor 1004 may be integrated into one semiconductor chip (CHIP). Additionally, the semiconductor chip (CHIP) may further include a storage interface 1005 and a network interface 1006.

The computer system 1000 may further include external memory 1013. The external memory 1013 may provide additional storage space for components within the computer system 1000, such as the main processor 1002, and may be provided in a detachable manner to the computer system 1000. For example, the processor 1002 may use the external memory 1013 together with the internal memory 1003 as a working memory.

Figure 11 is a block diagram showing a network system including a plurality of cameras.

Referring to FIG. 11, the network system 2000 includes a network 2005, a plurality of cameras 2110 to 21k0, a computer device 2050, an event management server 2200, a cloud database 2300, and a user terminal ( 2400). The event management server 2200, cloud database 2300, and user terminal 2400 are configured similarly to the event management server 120, cloud database 130, and user terminal 140 of FIG. 1 . Hereinafter, overlapping descriptions will be omitted.

A plurality of cameras, such as the first to k cameras 2110 to 21k0, may be installed in a specific area, such as inside a building, outside a building, or in a park. Furthermore, a computer device 2050 that communicates with the first to k cameras 2110 to 21k0 may be further provided. The computer device 2050 can control the first to k cameras 2110 to 21k0. In embodiments, the computer device 2050 may include and execute dedicated software for controlling the first to k cameras 2110 to 21k0. In embodiments, the computer device 2050 may be installed in a location adjacent to the first to kth cameras 2110 to 21k0. In embodiments, the computer device 2050 may include at least one of a wired communication function and a wireless communication function capable of communicating with the first to k cameras 2110 to 21k0.

At least one of the first to k cameras 2110 to 21k0 may be the image capture system 200 of FIG. 2, and accordingly performs the operations of the image capture system 200 described with reference to FIGS. 3 to 9. can do. Hereinafter, for convenience of explanation, it is assumed that the first camera 2110 among the first to k cameras 2110 to 21k0 functions as the image capturing system 200 of FIG. 2.

FIG. 12 is a flowchart showing operations of the first to k cameras and the computer device of FIG. 11 .

Referring to FIGS. 11 and 12 , in step S710, a privacy level is selected in the first camera 2110. The first camera 2110 can control capturing of images and operations related to the captured images according to the selected privacy level.

In step S720, the first camera 2110 notifies the computer device 2050 of the selected privacy level.

In step S730, the computer device 2050 transmits control signals so that the second to k cameras 2120 to 21k0 operate in the privacy mode in response to receiving the selected privacy level. For example, the computer device 2050 may inform the second to kth cameras 2120 to 21k0 of the selected privacy level.

In step S740, the second to k cameras (2120 to 21k0) enter the privacy mode. The second to k cameras 2120 to 21k0 operate in a privacy mode, but can control the capture of images and operations related to the captured images in the same way as the first camera 2110 according to the selected privacy level.

Accordingly, if only one camera among the first to k cameras 2110 to 21k0 includes the resources required to select one of the plurality of privacy levels, the remaining cameras may operate in the privacy mode even if they do not include such resources. It can operate as . For example, the first camera 2110 has relatively high performance and storage space, and although the remaining second to k cameras (2120 to 21k0) have lower performance and storage space, the second to k cameras (2120 to 21k0) have lower performance and storage space. The k cameras 2120 to 21k0 can enter the privacy mode like the first camera 2110 and further operate under a commonly selected privacy level.

Although specific embodiments and application examples are described herein, they are provided only to facilitate a more general understanding of the present invention, and the present invention is not limited to the above embodiments, and may be used by those of ordinary skill in the field to which the present invention pertains. Various modifications and variations are possible from this substrate.

Accordingly, the spirit of the present invention should not be limited to the described embodiments, and the scope of the patent claims described below as well as all modifications that are equivalent or equivalent to the scope of this patent claim shall fall within the scope of the spirit of the present invention. .

200: Video recording system
210: Image processing device
211: image sensor
212: Image processor
213: storage interface
214: network interface
215: main processor
216: Face detector
217: Privacy level controller

Claims (15)

a storage medium configured to store a plurality of image information sets each corresponding to a plurality of reference images and a plurality of privacy levels each corresponding to the plurality of image information sets; and
In privacy mode, it includes an image processing device that selectively controls the capture of an image and operations related to the captured image according to the selected privacy level,
The image processing device detects a face image from the captured image and selects a privacy level corresponding to an image information set that matches the detected face image among the plurality of privacy levels. It is configured to
The image processing device is configured to acquire facial images according to shooting of the image, generate the plurality of image information sets in association with the acquired facial images, and store the plurality of image information sets in the storage medium. composed,
Operations related to the captured image include transmitting the captured image to an external device,
The image processing device stops capturing the image when a first privacy level is selected among the plurality of privacy levels,
The image processing device stops transmitting the captured image to the external device when a second privacy level is selected among the plurality of privacy levels. delete According to claim 1,
The image processing device determines the privacy level corresponding to the image information set according to the number of face images associated with each image information set among the acquired facial images, thereby determining the privacy levels corresponding to the plurality of image information sets. An imaging system configured to determine and store the determined privacy levels in the storage medium. According to claim 3,
When the image processing device receives a user input indicating a privacy level corresponding to a selected image information set among the plurality of image information sets, the user inputs the privacy level stored in association with the selected image information set. An imaging system configured to be modified according to. According to claim 1,
The detected facial image includes first and second facial images,
The first and second facial images respectively match first and second image information sets among the plurality of image information sets,
The first and second image information sets correspond to first and second privacy levels, respectively, of the plurality of privacy levels,
The image processing device is configured to select one of the first and second privacy levels with a higher priority, and the priorities of the first and second privacy levels are determined in advance. ◈Claim 6 was abandoned upon payment of the setup registration fee.◈ According to claim 1,
Includes more communicators,
The image processing device is configured to transmit event information to an external device through the communicator when there is no image information set that matches the detected face image among the plurality of image information sets. According to claim 1,
The image processing device operates in the privacy mode when there is a set of image information that matches the detected face image among the plurality of image information sets, and returns to the privacy mode a predetermined time after starting to operate in the privacy mode. An imaging system configured to terminate. delete delete ◈Claim 10 was abandoned upon payment of the setup registration fee.◈ According to claim 1,
non-volatile storage media; and
Includes more communicators,
The operations related to the captured image include transmitting the captured image to an external device through the communicator. ◈Claim 11 was abandoned upon payment of the setup registration fee.◈ According to claim 10,
The operations include at least one of a first operation of storing the captured image in the non-volatile storage medium and a second operation of storing the captured image in an external storage medium through the communicator,
The image processing device stops the first operation and the second operation when a third privacy level is selected among the plurality of privacy levels. ◈Claim 12 was abandoned upon payment of the setup registration fee.◈ According to claim 1,
It further includes a communicator that provides communication with an external device,
The image processing device is configured to notify the external device of the selected privacy level through the communicator. Including a plurality of video recording devices connected to a network,
The first image capture device among the plurality of image capture devices is,
a communicator providing communication through the network;
a storage medium configured to store a plurality of image information sets each corresponding to a plurality of reference images and a plurality of privacy levels each corresponding to the plurality of image information sets; and
In privacy mode, it includes an image processing device that selectively controls the capture of an image and operations related to the captured image according to the selected privacy level,
The image processing device detects a face image from the captured image and selects a privacy level corresponding to an image information set that matches the detected face image among the plurality of privacy levels. It is configured to
At least one second image capture device among the plurality of image capture devices is in a privacy mode according to the privacy level selected by the first image capture device when one of the plurality of privacy levels is selected by the first image capture device. Enter and
Operations related to the captured image include transmitting the captured image to an external device,
The image processing device of the first image capture device stops capturing the image when a first privacy level is selected among the plurality of privacy levels,
The network system wherein the image processing device of the first image capture device stops transmitting the captured image to the external device when a second privacy level is selected among the plurality of privacy levels. ◈Claim 14 was abandoned upon payment of the setup registration fee.◈ According to claim 13,
Further comprising a computer device that communicates with the plurality of image capture devices through the network,
The first image capturing device notifies the selected privacy level to the computer device through the communicator,
A network system wherein the computer device controls the second image capture device to operate in the privacy mode according to the selected privacy level. ◈Claim 15 was abandoned upon payment of the setup registration fee.◈ According to claim 13,
A network system in which the second image capture device selectively controls capture of the image and operations related to the captured image in the same manner as the first image capture device in the privacy mode.

Images