KR102664027B1 - Camera to analyze video based on artificial intelligence and method of operating thereof - Google Patents

Abstract

In a method of operating any one of artificial intelligence cameras capable of communicating with each other through a network, each of the artificial intelligence cameras includes an artificial intelligence processor configured to perform image analysis using a deep learning method, and the operating method includes: Step of forming a cluster for video analysis with other artificial intelligence cameras by exchanging registration information with other artificial intelligence cameras through the network; registering the registration information of normal cameras with other artificial intelligence cameras so that normal cameras can be accessed through the network A step of sharing with a camera, when one of the normal cameras is assigned, a step of accessing the assigned normal camera and receiving an image taken by the assigned normal camera, a step of analyzing the received image using an artificial intelligence processor. , and transmitting the result information according to the analysis of the received video to an external device through a network along with the identifier of the assigned normal camera.

Description

Camera that analyzes video based on artificial intelligence and its operating method {CAMERA TO ANALYZE VIDEO BASED ON ARTIFICIAL INTELLIGENCE AND METHOD OF OPERATING THEREOF}

The present invention relates to a camera that can capture images and a method of operating the same, and more specifically, to a camera that analyzes images based on artificial intelligence and a method of operating the same.

In an image capture system that includes cameras connected to external equipment through a communication network, the cameras communicate with external equipment such as host devices and user terminals, but the cameras also communicate with each other using a P2P (peer to peer) method. You may. Nevertheless, in the past, cameras primarily provided services by communicating with external equipment. In this case, communication between the cameras is generally not performed so that each of the cameras can operate individually.

Meanwhile, cameras that include artificial intelligence functions are being developed. Artificial intelligence technology consists of deep learning (machine learning) and element technologies using deep learning. Deep learning is an algorithm that classifies/learns the characteristics of input data on its own, and elemental technology is a technology that utilizes deep learning to mimic the functions of the human brain such as perception and judgment, including visual understanding, inference/prediction, and knowledge expression. May include technologies. Visual understanding is a technology that recognizes and processes objects like human vision, and can include object recognition, object tracking, image search, person recognition, scene understanding, spatial understanding, and image improvement. Inference/prediction is a technology that judges information to make logical inferences and predictions, and may include knowledge/probability-based reasoning, optimization prediction, preference-based planning, and recommendations. Knowledge expression is a technology that automatically processes human experience information into knowledge data, and can include knowledge construction (data creation/classification), knowledge management (data utilization), etc.

The content described above is only intended to help understand the background technology of the technical ideas of the present invention, and therefore, it cannot be understood as content corresponding to prior art known to those skilled in the art of the present invention.

Embodiments of the present invention relate to an artificial intelligence camera and a method of operating the same that can efficiently expand the scope of image analysis based on artificial intelligence.

In a method of operating any one of artificial intelligence cameras capable of communicating with each other through a network according to an embodiment of the present invention, each of the artificial intelligence cameras is an artificial intelligence processor configured to perform image analysis using a deep learning method. Including, the operating method includes exchanging registration information with other artificial intelligence cameras through the network, thereby configuring a cluster for image analysis with the other artificial intelligence cameras; sharing registration information of the normal cameras with the other artificial intelligence cameras so that the normal cameras can be accessed through the network; When one of the normal cameras is assigned, accessing the assigned normal camera and receiving an image captured by the assigned normal camera; Analyzing the received image using the artificial intelligence processor; and transmitting result information according to the analysis of the received video to an external device through the network along with the identifier of the assigned normal camera.

The operating method includes receiving registration information of an additional artificial intelligence camera; And it may further include accessing the additional artificial intelligence camera according to the received registration information and adding the additional artificial intelligence camera to the cluster.

The step of configuring the cluster includes determining one of the artificial intelligence cameras as a master node of the cluster by communicating with the other artificial intelligence cameras, wherein the assigned normal camera responds to a command from the master node. It can be decided accordingly.

Receiving the captured image may include receiving an event alarm signal generated from the assigned normal camera; and requesting the assigned normal camera to provide the captured image in response to the event alarm signal.

The step of configuring the cluster includes determining one of the artificial intelligence cameras as a master node of the cluster by communicating with the other artificial intelligence cameras, wherein the event alarm signal can be received through the master node. there is.

The step of receiving the captured image includes changing the settings of the assigned normal camera to a predetermined setting value in response to the event alarm signal before requesting the assigned normal camera to provide the captured image. It may further include.

In the step of receiving the image captured by the assigned normal camera, depending on whether the probability according to deep learning is greater than or equal to a threshold when analyzing the received image using the artificial intelligence processor, the assigned normal camera The length of the video received from can be variably adjusted.

The operating method includes communicating status messages with the other artificial intelligence camera; When the other artificial intelligence camera is determined to be abnormal based on the status messages, accessing another normal camera assigned to the other artificial intelligence camera and additionally receiving images captured by the other normal camera; analyzing the additionally received image using the artificial intelligence processor; And it may further include transmitting result information according to the analysis of the additionally received image to the external device through the network along with the identifier of the other normal camera.

The artificial intelligence camera further includes an image sensor, and the operating method includes capturing an image through the image sensor; And it may further include analyzing the captured image using the artificial intelligence processor.

The transmitting to the external device includes transmitting the captured image, first metadata, and second metadata to the external device, wherein the first metadata is used in the analysis of the captured image. The second metadata may include the result information according to the analysis of the received image and the identifier of the assigned normal camera.

The step of configuring the cluster includes receiving the registration information of the other artificial intelligence camera from a user terminal; And it may include configuring the cluster by communicating with the other artificial intelligence camera using the received registration information.

The sharing of the registration information of the normal cameras with the other artificial intelligence cameras includes: receiving the registration information of the normal cameras from a user terminal; And it may include providing the received registration information to the other artificial intelligence camera.

Another aspect of the present invention relates to a camera. A camera according to an embodiment of the present invention includes an optical system; a first processor configured to process an image generated according to light received through the optical system; a second processor that operates in response to control of the first processor and is configured to analyze the processed image based on a deep learning method; and a network interface that provides communication through a network, wherein the first processor exchanges registration information with an external artificial intelligence camera through the network, thereby forming a cluster for image analysis with the external artificial intelligence camera. In order to access the normal cameras through the network, the registration information of the normal cameras is shared with the external artificial intelligence camera, and when one of the normal cameras is assigned, the assigned normal camera is accessed. Receive an image captured by the assigned normal camera, command the second processor to analyze the received image, and provide result information according to the analysis of the received image together with the identifier of the assigned normal camera. It is configured to transmit to an external device through the network.

When registration information of an additional artificial intelligence camera is received, the first processor may be configured to access the additional artificial intelligence camera according to the received registration information and add the additional artificial intelligence camera to the cluster.

The first processor is configured to communicate with the external artificial intelligence camera to determine one of the cameras in the cluster as the master node of the cluster, and the assigned normal camera can be determined according to a command from the master node. .

The first processor may be configured to change the settings of the assigned normal camera to a predetermined value before requesting the assigned normal camera to provide the captured image.

The first processor is configured to variably adjust the length of the image received from the assigned normal camera depending on whether the probability according to deep learning is greater than a threshold when the second processor analyzes the received image. It can be.

According to embodiments of the present invention, an artificial intelligence camera and a method of operating the same are provided that can efficiently expand the scope of image analysis based on artificial intelligence.

1 is a block diagram showing an image capturing system according to an embodiment of the present invention.
FIG. 2 is a block diagram showing an example of one of the artificial intelligence cameras of FIG. 1.
FIG. 3 is a block diagram showing an embodiment of the main processor of FIG. 2.
Figure 4 is a block diagram showing an embodiment of the artificial intelligence processor of Figure 2.
Figure 5 is a diagram conceptually showing an example of the format of output data from an artificial intelligence camera.
Figure 6 is a flowchart showing a method of analyzing images captured by an artificial intelligence camera.
Figure 7 is a flowchart showing a method of operating an artificial intelligence camera to configure a cluster that can interact with normal cameras according to an embodiment of the present invention.
Figure 8 is a flowchart showing an example of step S210 of Figure 7.
Figure 9 is a flowchart showing an example of step S220 of Figure 7.
Figure 10 is a table showing cluster information stored and shared with artificial intelligence cameras after forming a cluster.
Figure 11 is a table showing normal camera information stored and shared with artificial intelligence cameras after forming a cluster.
Figure 12 is a flowchart showing an example of a method for adding an artificial intelligence camera to a cluster that can interact with normal cameras.
Figure 13 is a flowchart showing a method of analyzing images from a normal camera in an artificial intelligence camera included in a cluster according to an embodiment of the present invention.
FIG. 14 is a flowchart showing an example of step S420 of FIG. 12.
Figure 15 is a flowchart showing a method of analyzing images from a normal camera in an artificial intelligence camera included in a cluster according to another embodiment of the present invention.

Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the attached drawings. It should be noted that in the following description, only the parts necessary to understand the operation according to the present invention will be described, and the description of other parts will be omitted in order to not obscure the gist of the present invention. Additionally, the present invention is not limited to the embodiments described herein and may be embodied in other forms. However, the embodiments described herein are provided to explain in detail enough to enable those skilled in the art to easily implement the technical idea of the present invention.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected," but also the case where it is "indirectly connected" with another element in between. . The terminology used herein is for the purpose of describing specific embodiments and is not intended to limit the invention. Throughout the specification, when a part is said to “include” a certain element, this means that it may further include other elements rather than excluding other elements, unless specifically stated to the contrary. “at least one of X, Y, and Z”, and “at least one selected from the group consisting of It can be interpreted as any combination (e.g., XYZ, XYY, YZ, ZZ). As used herein, “and/or” includes any combination of one or more of the constituents.

1 is a block diagram showing an image capturing system according to an embodiment of the present invention.

An imaging system may include a plurality of devices, servers, and/or software components operative to perform methods according to embodiments described herein. The devices and servers shown in FIG. 1 may be deployed in different ways, and the operations and services provided by such devices and/or servers may be combined or separated to perform the embodiments described herein. and may be performed by more or fewer devices and servers. One or more devices and/or servers may be run by the same or different entities, for example companies.

Referring to FIG. 1, the image capturing system 100 includes a network 50, a user terminal 110, a plurality of artificial intelligence cameras 121 to 123, and first to nth normal cameras 131. ~13n), and may include a video management server 140.

Network 50 may include at least one of a public network, at least one private network, a wired network, a wireless network, another suitable type of network, and combinations thereof. Each of the components within the imaging system 100 may include at least one of a wired communication function and a wireless communication function, and thus may communicate with each other through the network 50 .

The user terminal 110 can communicate with a plurality of artificial intelligence cameras 121 to 123, first to nth normal cameras 131 to 13n, and the video management server 140 through the network 50. . The user terminal 110 can receive images being captured in real time from each of the plurality of artificial intelligence cameras 121 to 123 and the first to nth normal cameras 131 to 13n, and display the received images. there is. Additionally, the user terminal 110 may access the video management server 140, receive images stored in the video management server 140, and display the received images. At this time, the images stored in the video management server 140 may be images captured by a plurality of artificial intelligence cameras 121 to 123 and the first to nth normal cameras 131 to 13n.

The video management server 140 further stores metadata that may include various types of information, such as motion detection, face detection, intrusion detection, gun sound detection, object/pattern information, and similarity information of objects, in relation to the image. However, embodiments of the present invention are not limited thereto. For example, images and metadata associated with images may be stored in different databases.

Metadata can be provided by artificial intelligence cameras. The user terminal 110 receives metadata from the video management server 140, displays information indicated by the received metadata, and receives the corresponding video from the video management server 140 in response to a user input for selecting metadata. You can receive and display the received video.

In embodiments, the user terminal 110 includes a computer device, and the computer device includes a plurality of artificial intelligence cameras 121 to 123 connected to the network 50, and first to nth normal cameras 131 to 13n. , and may include application applications for accessing components such as the video management server 140, receiving video therefrom, and displaying the received video.

A plurality of artificial intelligence cameras 121 to 123 may be provided to the image capture system 100. In Figure 1, it is illustrated that three artificial intelligence cameras 121 to 123 are provided. Each of the first to third artificial intelligence cameras 121 to 123 includes at least one artificial intelligence processor configured to perform image analysis based on artificial intelligence. Each of the first to third artificial intelligence cameras 121 to 123 captures images, analyzes the captured images through an artificial intelligence processor, and provides the captured images and video analysis results to the video management server 140. You can. Additionally, each of the first to third artificial intelligence cameras can record audio, analyze the recorded audio through an artificial intelligence processor, and provide the recorded audio and audio analysis results to the video management server 140. Hereinafter, for convenience of explanation, embodiments of the present invention will be described focusing on operations for image analysis. However, this is for convenience of explanation, and the technical idea of the present invention can also be applied to audio analysis.

Each of the first to third artificial intelligence cameras 121 to 123 receives an image from at least one of the first to nth normal cameras 131 to 13n, and analyzes the received image through an artificial intelligence processor, The video analysis results may be provided to the video management server 140. At this time, the normal camera may be a non-AI camera. For example, a normal camera may include a processor configured to perform conventional event detection algorithms, such as motion detection, face detection, and sound detection, rather than artificial intelligence. , on the other hand, may not include an artificial intelligence processor.

The first to third artificial intelligence cameras (121 to 123) exchange registration information with each other to form a cluster (CLST) for image analysis, and each normal camera is connected to the artificial intelligence cameras (121 to 123) in the cluster (CLST). ) can be assigned to any one of the following. At this time, the registration information may include information for accessing the camera, such as the artificial intelligence camera's URL (uniform resource locator) address, IP (internet protocol) address, port number, and camera login account. The registration information of each artificial intelligence camera may be received from the user terminal 110 to any one of the first to third artificial intelligence cameras (121 to 123), and the first to third artificial intelligence cameras (121 to 123) ) can be shared by. In embodiments, the first to third artificial intelligence cameras 121 to 123 may periodically synchronize their registration information.

Which normal camera to assign to each of the first to third artificial intelligence cameras 121 to 123 can be determined according to various methods. For example, the first to third artificial intelligence cameras 121 to 123 may inform each other of their available resources, and allocation to each normal camera may be performed with reference to such available resources. For example, the first to third artificial intelligence cameras 121 to 123 communicate with each other to determine which artificial intelligence camera will function as a master node, and the master node may perform a task scheduling function. The master node can assign each normal camera to one of the first to third artificial intelligence cameras (121 to 123) by referring to the available resources of each of the first to third artificial intelligence cameras (121 to 123). .

In this way, by configuring a cluster (CLST) including the first to third artificial intelligence cameras (121 to 123) and analyzing the images of the first to nth normal cameras (131 to 13n), each normal camera It can be assigned to any one of the first to third artificial intelligence cameras (121 to 123), for example, by referring to available resources, and thus to the images of the first to nth normal cameras (131 to 13n). Analysis can be efficiently performed using the first to third artificial intelligence cameras 121 to 123 without providing a server including a separate artificial intelligence function. Accordingly, without a server containing a separate artificial intelligence function, image analysis based on artificial intelligence can be performed on not only the images captured by the first to nth artificial intelligence cameras 121 to 123, but also the first to nth normal cameras. It can be extended to images captured by fields 131 to 13n. Therefore, an artificial intelligence camera and an image capture system including the same that can efficiently expand the scope of image analysis based on artificial intelligence can be provided.

FIG. 2 is a block diagram showing an example of one of the artificial intelligence cameras of FIG. 1. FIG. 3 is a block diagram showing an embodiment of the main processor of FIG. 2. Figure 4 is a block diagram showing an embodiment of the artificial intelligence processor of Figure 2.

Referring to FIG. 2, the artificial intelligence camera 200 includes an optical system 205, an image sensor 210, RAM (220, Random Access Memory), EEPROM (230, Electrically Erasable and Programmable Read Only Memory), and storage ( 240), a main processor 250, a network interface 260, and an artificial intelligence (AI) processor 270.

The optical system 205 optically processes light from the subject.

The image sensor 210 may operate in response to control from the main processor 250. The image sensor 210 is configured to convert the optical signal received through the lens 205 into an electrical signal and digitize the converted electrical signal to generate an image. For example, the image sensor 210 may include an analog-to-digital converter configured to convert analog image signals into digital image data.

RAM 220 is connected to the main processor 250. RAM 220 may temporarily store images processed by the main processor 250. RAM 220 may be used as a buffer memory. RAM 220 may be used as a working memory of the main processor 250. Electrically Erasable and Programmable Read Only Memory (EEPROM) 230 can store program codes and/or instructions necessary for the operation of the main processor 250. The storage 240 may store setting data required for operations of the main processor 250. Additionally, the storage 240 may further store program codes and/or instructions to be executed by the main processor 250. These program codes and/or instructions may be loaded into RAM 220 from storage 240 and executed by main processor 250. At least some of the operations of the main processor 250, which will be described below, may be performed by the main processor 250 by executing these program codes and/or instructions. In embodiments, storage 240 may include a non-volatile storage medium, such as flash memory.

The main processor 250 controls overall operations of the artificial intelligence camera 200. The main processor 250 may communicate with components connected to the network 50 (see FIG. 1) through the network interface 260. The main processor 250 is configured to appropriately process the image received from the image sensor 210, and can transmit the processed image to the user terminal 110 through the network interface 260 in live-view. Additionally, the main processor 250 may upload the processed video to the video management server 140 through the network interface 260. Furthermore, the main processor 250 may receive audio data received through a microphone (not shown), appropriately process the received audio data, and transmit the processed audio data to external equipment through the network interface 260. . In this case, audio data and video data may constitute multimedia data.

Referring to FIG. 3, the main processor 250 includes a core processor 251, a data formatter 252, a data converter 253, a resizer 254, a Moving Picture Experts Group (MPEG) encoder 255, and Includes a Joint Photographic Experts Group (JPEG) encoder (256).

The core processor 251 controls overall operations of the artificial intelligence camera 200 (see FIG. 2). Additionally, the core processor 251 controls the data formatter 252, data converter 253, resizer 254, MPEG encoder 255, and JPEG encoder 256. The data formatter 252 stores the image from the image sensor 210 (see FIG. 2) in the RAM 220, and the data converter 253 converts the image into red (R), green (G), and blue (B). The image data in the format is converted into image data in the luminance (Y) and chrominance (Cb, Cr) formats, and the resizer 254 can convert the resolution of the image data. The MPEG encoder 255 is a video encoder that compresses video data, and the JPEG encoder 256 is a still image encoder that can compress still image data. In this way, the main processor 250 is configured to process the image received from the image sensor 210.

In embodiments, components for image processing such as the data formatter 252, the data converter 253, the resizer 254, the MPEG encoder 255, and the JPEG encoder 256 are hardware, software, and firmware, respectively. , and their combinations, and can be combined with each other or separated into more components. If at least one of the data formatter 252, data converter 253, resizer 254, MPEG encoder 255, and JPEG encoder 256 is implemented as software, the software may be implemented as storage 240 of FIG. 2. It may be stored in a computer-readable storage medium such as, and may be executed by the core processor 251. In addition, at least some of the components for image processing, such as the data formatter 252, the data converter 253, the resizer 254, the MPEG encoder 255, and the JPEG encoder 256, perform image processing. It can be implemented through one dedicated processor, such as a unit.

Referring again to FIG. 2, the main processor 250 may command the artificial intelligence processor 270 to analyze the processed image. The artificial intelligence processor 270 is configured to analyze images processed based on artificial intelligence. The video analysis results are temporarily stored in the RAM 220, for example, and the main processor 250 sends the video analysis results along with the video to the user terminal 110 or the video management server 140 through the network interface 260. Can be transmitted.

Referring to FIG. 4, the artificial intelligence processor 270 may include a data learning unit 271 and a data analysis unit 272. The data learning unit 271 may include a deep learning framework, and thus may perform neural network algorithms such as CNN (Convolutional neural network) and vision algorithms such as SIFT (Scale Invariant Feature Transform). The data learning unit 271 can learn various image analysis standards, such as object/pattern recognition and similarity checking of objects. The data learning unit 271 can learn and infer standards regarding which data to use for image analysis. The data learning unit 271 can learn standards for image analysis by acquiring data to be used for learning and applying the acquired data to a data analysis model.

The data analysis unit 272 may perform analysis on the input image in response to a request from the main processor 250 (see FIG. 2). The data analysis unit 272 may perform image analysis using the learned data recognition model. The data analysis unit 272 acquires data according to the standards set by learning, performs image analysis by using a data analysis model that uses the acquired data as input, and outputs result information according to the analysis. . Additionally, the result information of image analysis can be used to update the data analysis model.

The data learning unit 271 and the data analysis unit 272 may each be implemented through hardware, software, firmware, and a combination thereof, and may be combined with each other or separated into more components. At least one of the data learning unit 271 and the data analysis unit 272 may be implemented in the form of a dedicated hardware chip for artificial intelligence, or may be implemented through a general-purpose processor or a graphics-specific processor. When at least one of the data learning unit 271 and the data analysis unit 272 is implemented as software, the software may be stored in a computer-readable storage medium such as the storage 240 of FIG. 2, and the general-purpose processor or It can be executed by a processor, such as a dedicated graphics processor.

The data learning unit 271 and the data analysis unit 272 may be mounted on one electronic device, or may be mounted on separate electronic devices. For example, the data learning unit 271 may be included in an external device, and the data analysis unit 272 may be included in the artificial intelligence camera 200 (see FIG. 2). In this case, the data learning unit 271 and the data analysis unit 272 may communicate by wire or wirelessly, and the data learning unit 271 may provide the built model information to the data analysis unit 272, Data input to the data analysis unit 272 may be provided to the data learning unit 271 as additional learning data.

Referring again to FIG. 2, the main processor 250 configures a cluster (CLST, see FIG. 1) for image analysis by exchanging registration information with other artificial intelligence cameras, and uses the first to nth normal cameras ( The registration information of the first to nth normal cameras (131 to 13n) can be shared with other artificial intelligence cameras so that the cameras (131 to 13n) can be accessed. Communications with other artificial intelligence cameras and the first to nth normal cameras 131 to 13n are performed through the network interface 260. When any one of the first to nth normal cameras 131 to 13n is assigned, the main processor 250 may control the artificial intelligence processor 270 to analyze the image captured by the assigned normal camera. .

Figure 5 is a diagram conceptually showing an example of the format of output data from an artificial intelligence camera.

2 and 5, the main processor 250 outputs output data (DOUT) including video (VD), audio (AD), first metadata (META1), and second metadata (META2). It can be transmitted to the user terminal 110 or the video management server 140 through the network interface 260. Video (VD) and audio (AD) are generated by the main processor 250 as described with reference to FIG. 2. The first metadata (META1) may include analysis result information of the artificial intelligence processor 270 for video (VD) and/or audio (AD). The second metadata (META2) may include information on the results of analysis by the artificial intelligence processor 270 of the video and/or audio received from the normal camera.

The second metadata (META2) may further include an identifier of the corresponding normal camera. The identifier may include various types of data that can identify the normal camera, such as the normal camera ID and IP address. Accordingly, the user terminal 110 or the video management server 140 receiving the output data (DOUT) can determine that the second metadata (META2) included in the output data (DOUT) corresponds to a certain normal camera. there is.

More details regarding the operations of main processor 250 are described below.

Figure 6 is a flowchart showing a method of analyzing images captured by an artificial intelligence camera.

Referring to FIGS. 2 and 6 , in step S110, an image is captured through the image sensor 210. In step S120, the captured image is analyzed using the artificial intelligence processor 270. The main processor 250 may process images captured through the image sensor 210 and command the artificial intelligence processor 270 to analyze the processed images. The main processor 250 stores the resulting information according to the analysis of the captured video, audio, and video into the video (VD) field, the audio (AD) field, and the first metadata (META1) of the output data (DOUT) shown in FIG. 5. ) can be included in each field.

Figure 7 is a flowchart showing a method of operating an artificial intelligence camera to configure a cluster that can interact with normal cameras according to an embodiment of the present invention. Figure 8 is a flowchart showing an example of step S210 of Figure 7. Figure 9 is a flowchart showing an example of step S220 of Figure 7.

Referring to FIG. 7 along with FIGS. 1 and 2 , in step S210, the artificial intelligence camera 200 forms a cluster for image analysis with other artificial intelligence cameras by exchanging registration information with other artificial intelligence cameras. The first to third artificial intelligence cameras 121 to 123 can communicate with each other through the network 50 as described with reference to FIG. 1, and each artificial intelligence camera transmits registration information to other artificial intelligence cameras. And by receiving registration information from other artificial intelligence cameras, a cluster (CLST) including artificial intelligence cameras 121 to 123 can be created.

Registration information for each artificial intelligence camera may be provided by the user through the user terminal 110. Referring to FIG. 8, the artificial intelligence camera 200 receives registration information of another artificial intelligence camera from the user terminal 110 (S211) and communicates with the artificial intelligence camera corresponding to the received registration information, thereby A cluster (CLST) can be formed with intelligent cameras. For example, when the user terminal 110 accesses the first artificial intelligence camera 121 and provides registration information of the second and third artificial intelligence cameras 122 and 123, the first artificial intelligence camera 121 As accessing each of the second and third artificial intelligence cameras 122 and 123 using the provided registration information, the first to third artificial intelligence cameras 121 to 123 can share each other's registration information. there is. Accordingly, a cluster (CLST) including the first to third artificial intelligence cameras 121 to 123 may be created.

When creating a cluster (CLST), the first to third artificial intelligence cameras 121 to 123 of the cluster (CLST) may communicate with each other to negotiate to determine one of them as the master node. Accordingly, one of the first to third artificial intelligence cameras 121 to 123 may function as a master node, and the remaining artificial intelligence cameras may function as agent nodes. The master node configures each of the first to nth normal cameras 131 to 13n to perform image analysis using a deep learning method on the images of the first to nth normal cameras 131 to 13n. It can be assigned to any one of the third artificial intelligence cameras (121 to 123).

Referring again to FIGS. 1, 2, and 7, in step S220, the artificial intelligence camera 200 shares registration information of the first to nth normal cameras 131 to 13n with other artificial intelligence cameras. Accordingly, each of the first to third artificial intelligence cameras 121 to 123 can access the first to nth normal cameras 131 to 13n through the network 50. For example, the artificial intelligence camera 200 can provide registration information of a normal camera to another artificial intelligence camera, and can also receive registration information of a normal camera from another artificial intelligence camera. At this time, the registration information may include information for accessing the camera, such as the normal camera's URL (uniform resource locator) address, IP (internet protocol) address, port number, and camera login account.

Registration information for the normal camera may be provided by the user through the user terminal 110. Registration information of the first to nth normal cameras 131 to 13n may be shared with the first to third artificial intelligence cameras 121 to 123 according to various methods. Referring to FIG. 9, the artificial intelligence camera 200 receives registration information of the first to nth normal cameras 131 to 13n from the user terminal 110 (S221), and sends the received registration information to another artificial intelligence. It can be provided to the camera. For example, when the user terminal 110 provides registration information of the first to nth normal cameras 131 to 13n to the first artificial intelligence camera 121, the first artificial intelligence camera 121 provides the registration information. Information can be provided to the second and third artificial intelligence cameras 122 and 123.

Figure 10 is a table showing cluster information stored and shared with artificial intelligence cameras after forming a cluster.

After the cluster (CLST) is created, the first to third artificial intelligence cameras (121 to 123) within the cluster (CLST) can store and share cluster information (CLSTIF). 1 and 10, cluster information (CLSTIF) is an IP address corresponding to each of the identifiers (AICAM1, AICAM2, AICAM3) of the first to third artificial intelligence cameras 121 to 123, based on artificial intelligence. It may include the usage of the artificial intelligence processor and the usage of the main processor according to the processing of tasks.

Cluster information (CLSTIF) is collected by the master node among the first to third artificial intelligence cameras (121 to 123) and may be shared throughout the first to third artificial intelligence cameras (121 to 123). With reference to the usage of the artificial intelligence processor and the usage of the main processor in the cluster information (CLSTIF), the master node connects each of the first to nth normal cameras (131 to 13n) to the first to third artificial intelligence cameras (121 to 121). 123) can be assigned to any one of the following.

The user may access any one of the first to third artificial intelligence cameras 121 to 123 of the cluster CLST through the user terminal 110 and query at least part of the cluster information CLSTIF.

Figure 11 is a table showing normal camera information stored and shared with artificial intelligence cameras after forming a cluster. In FIG. 11, for convenience of explanation, it is illustrated that five normal cameras are provided in the image capture system.

The first to third artificial intelligence cameras (121 to 123) in the cluster (CLST) can store and share normal camera information (NRMCAMIF). 1 and 11, normal camera information (NRMCAMIF) may include an IP address and a detection event type corresponding to each of the identifiers (NRMCAM1 to NRMCAM5) of the first to fifth normal cameras 131 to 135. You can. In embodiments, the normal camera may include a processor configured to perform image analysis based on at least one of conventional event detection algorithms rather than artificial intelligence, and the normal camera information (NRMCAMIF) may be configured to detect the event employed in the normal camera. Additional information about the event type of the algorithm can be stored. For example, detection event types may include motion detection, face detection, sound detection, etc.

Figure 12 is a flowchart showing an example of a method for adding an artificial intelligence camera to a cluster that can interact with normal cameras.

Referring to FIG. 12 along with FIGS. 1 and 2 , in step S310, the artificial intelligence camera 200 may receive registration information of an additional artificial intelligence camera from the user terminal 110. In step S320, the artificial intelligence camera 200 accesses the additional artificial intelligence camera according to the received registration information and adds the artificial intelligence camera to the cluster (CLST). Additionally, the received registration information can be shared with artificial intelligence cameras in the cluster (CLST) including the artificial intelligence camera 200. For example, when registration information of an additional artificial intelligence camera input by the user is provided to the first artificial intelligence camera 121 through the user terminal 110, the first artificial intelligence camera 121 provides the provided registration information. It can be transmitted to the second and third artificial intelligence cameras (122, 123). Each of the first to third artificial intelligence cameras (121 to 123) accesses an additional artificial intelligence camera according to the provided registration information, and the first to third artificial intelligence cameras (121 to 123) and the additional artificial intelligence camera access each other. You can share registration information. Accordingly, the corresponding artificial intelligence camera can be added to the cluster (CLST).

In step S330, the artificial intelligence camera 200 may share the registration information of each normal camera with the added artificial intelligence camera. For example, the first to third artificial intelligence cameras 121 to 123 may share normal camera information (NRMCAMIF, see FIG. 11) with additional artificial intelligence cameras.

In this way, the resources of the cluster (CLST) can be expanded relatively easily, and therefore, without the provision or expansion of a server containing separate artificial intelligence functions, through the addition of an artificial intelligence camera that can capture images by itself. , it is possible to expand the number of normal cameras that can support video analysis based on artificial intelligence while expanding the scope of surveillance through video shooting.

Figure 13 is a flowchart showing a method of analyzing images from a normal camera in an artificial intelligence camera included in a cluster according to an embodiment of the present invention.

Referring to FIG. 13 along with FIGS. 1 and 2 , in step S410, any one of the first to nth normal cameras 131 to 13n may be assigned to the artificial intelligence camera 200. As described above, allocation may be performed by a master node among the first to third artificial intelligence cameras 121 to 123 in the cluster (CLST). At this time, the artificial intelligence camera 200 may be a master node or an agent node. When a normal camera is assigned, step S420 is performed.

In step S420, the artificial intelligence camera 200 accesses the assigned normal camera and receives an image captured by the assigned normal camera.

In step S430, the artificial intelligence camera 200 analyzes the received image using the artificial intelligence processor 270.

In step S440, the artificial intelligence camera 200 transmits the result information according to the analysis of the video to the video management server 140 along with the identifier of the assigned normal camera. The artificial intelligence camera 200 may include result information from video analysis and the identifier of the normal camera in the second metadata (META2) field of the output data (DOUT) shown in FIG. 5.

In this way, by analyzing the images of the first to nth normal cameras (131 to 13n) through the cluster (CLST) including the artificial intelligence cameras (121 to 123), each normal camera is an artificial intelligence camera (121). ~123), for example, by referring to available resources, and therefore analysis of the images of the first to nth normal cameras 131 to 13n can be performed using a server that includes a separate artificial intelligence function. It can be performed efficiently using the artificial intelligence cameras 121 to 123 already provided without providing. Accordingly, without a server containing a separate artificial intelligence function, image analysis based on artificial intelligence can be performed on not only the images captured by the first to nth artificial intelligence cameras 121 to 123, but also the first to nth normal cameras. It can be extended to images captured by fields 131 to 13n. Therefore, an image analysis method that can efficiently expand the scope of image analysis based on artificial intelligence can be provided.

Figure 14 is a flowchart showing an example of step S420 of Figure 12.

Referring to FIG. 14 along with FIGS. 1 and 2 , in step S421, the artificial intelligence camera 200 may receive an event alarm signal from any one of the assigned normal cameras 131 to 131. When an event alarm signal is received, step S422 may be performed.

A normal camera may be configured to perform conventional event detection algorithms, such as motion detection, face detection, and sound detection, rather than artificial intelligence, and may generate an event alarm signal in response to detection of an event. In embodiments, the artificial intelligence camera 200 may request the normal camera to provide an event alarm signal in response to detection of an event, in response to the assignment of the normal camera, and the normal camera may request the normal camera to provide an event alarm signal to the artificial intelligence camera 200. An alarm signal can be provided. In embodiments, a master node may request a normal camera to provide an event alarm signal in response to detection of an event, and the normal camera may provide an event alarm signal to the master node. In this way, if the artificial intelligence camera 200 is a master node, it can receive an event alarm signal from the normal camera. If the artificial intelligence camera 200 is an agent node, it can receive an event alarm signal through the master node.

When a normal camera detects an event, it can enter a special mode that supports image analysis through artificial intelligence (or deep learning). For example, a normal camera can change its internal settings to preset values. Detection of an event means that image analysis through artificial intelligence is required, and the settings can be changed to preset values to suit image analysis through artificial intelligence. An image suitable for being transmitted to the video management server 140 and provided to a user through a display may be different from an image suitable for image analysis through artificial intelligence. If a normal camera is set to capture images suitable for image analysis through artificial intelligence, a user watching images received through the image management server 140 may feel uncomfortable. For this reason, the normal camera may be set to capture images suitable to be provided to the user. On the other hand, when the artificial intelligence camera 200 receives an image suitable to be provided to a user and performs image analysis, it is required to perform preprocessing on the image to make it suitable for image analysis. During the preprocessing process, a lot of noise may occur in the image and it may take a lot of time. For this reason, the settings of the normal camera are changed to values suitable for image analysis through artificial intelligence, and the artificial intelligence camera 200 can receive images captured under the corresponding settings from the normal camera. Accordingly, the artificial intelligence camera 200 can quickly and accurately perform image analysis without pre-processing the image or through a reduced pre-processing process. In other words, setting values such as sharpness or exposure can be adjusted to suit the artificial intelligence camera 200, making the image quality of the normal camera suitable for artificial intelligence (or deep learning). there is. The normal camera can change setting values in response to a setting value control request from the artificial intelligence camera 200, and can also change setting values in response to detection of an event. Alternatively, the setting values may be changed in response to control of the video management server 140. Entry into this special mode of the normal camera can be performed before step S422.

A normal camera can change its internal settings to the special mode values only while shooting the frames required for video analysis through artificial intelligence, and then restore the settings to their previous settings. Alternatively, the artificial intelligence camera 200 may control the normal camera to restore the settings when the image analysis of the artificial intelligence camera 200 ends.

In step S422, in response to the event alarm signal, the artificial intelligence camera 200 requests the normal camera to provide the captured image.

In step S423, the artificial intelligence camera 200 may receive an image from the normal camera and analyze the received image using the artificial intelligence processor 270.

As another example, the artificial intelligence camera 200 may constantly access the assigned normal camera to receive images and analyze the received images.

When analyzing the image of the normal camera, the artificial intelligence camera 200 determines the length of the image received from the normal camera through steps S422 and S423 depending on whether the probability according to deep learning (or artificial intelligence) is greater than the threshold. It can be controlled variably. For example, the artificial intelligence processor 270 may analyze images through deep learning and output a probability according to deep learning. If this probability is greater than or equal to a threshold, it may mean that image analysis is complete. Since additional image analysis is not required when the probability according to deep learning is greater than the threshold, the artificial intelligence camera 200 and/or the main processor 250 selects the normal camera depending on whether the probability according to deep learning is greater than the threshold. The length of the received video can be variably adjusted. The artificial intelligence camera 200 can receive an image of a relatively short length or an image of a relatively long length depending on whether the probability according to deep learning is greater than or equal to a threshold. For example, if additional images are needed, the artificial intelligence camera 200 can request and receive additional images. In this case, the length of the additional images, the number of times to request additional images, etc. are variable depending on the probability according to deep learning. can be adjusted.

When the artificial intelligence camera 200 receives a still image from the normal camera, additional still images may be requested depending on whether the probability according to deep learning is greater than the threshold, and the artificial intelligence camera 200 may receive a dynamic image from the normal camera. When receiving, you can request to stop transmitting dynamic images depending on whether the probability according to deep learning exceeds the threshold.

In this way, the artificial intelligence camera 200 can variably control the length of the image received from the normal camera depending on whether the probability according to deep learning is greater than or equal to the threshold. Accordingly, the time to change or restore the settings of the normal camera can also be accelerated.

Figure 15 is a flowchart showing a method of analyzing images from a normal camera in an artificial intelligence camera included in a cluster according to another embodiment of the present invention.

Referring to FIG. 15 along with FIGS. 1 and 2, in step S510, the artificial intelligence camera 200 transmits and receives status messages with other artificial intelligence cameras to detect abnormalities among the artificial intelligence cameras 121 to 123 in the cluster (CLST). Determine whether an artificial intelligence camera exists. Artificial intelligence cameras 121 to 123 may periodically exchange status messages.

In step S520, the artificial intelligence camera 200 performs step S530 when the artificial intelligence camera in the cluster (CLST) is abnormal. Hereinafter, for convenience of explanation, it is assumed that the artificial intelligence camera 200 is the master node of the cluster (CLST).

Step S530 may be performed by the master node. In step S530, the artificial intelligence camera 200 may assign the normal camera assigned to the abnormal artificial intelligence camera to another artificial intelligence camera. The master node refers to the usage of the artificial intelligence processor and the usage of the main processor in the cluster information (CLSTIF, see FIG. 10) and allocates the normal camera to one of the first to third artificial intelligence cameras 121 to 123. can do. Hereinafter, for convenience of explanation, it is assumed that the corresponding normal camera is assigned to the artificial intelligence camera 200.

In step S540, the artificial intelligence camera 200 accesses the assigned normal camera and receives images. In step S550, the artificial intelligence camera 200 analyzes the received image using the artificial intelligence processor 270. In step S560, the artificial intelligence camera 200 transmits the result information according to the analysis of the video to the video management server 140 along with the identifier of the corresponding normal camera.

In this way, abnormal resources within the cluster (CLST) can be relatively easily excluded from the cluster (CLST), and therefore image analysis based on artificial intelligence can be stably performed on images from normal cameras. For example, the time that AI-based video analysis is not available for normal cameras can be reduced or minimized.

Although specific embodiments and application examples are described herein, they are provided only to facilitate a more general understanding of the present invention, and the present invention is not limited to the above embodiments and can be understood by those skilled in the art. Various modifications and variations are possible from this substrate.

Accordingly, the spirit of the present invention should not be limited to the described embodiments, and the scope of the patent claims described later as well as all things that are equivalent or equivalent to the scope of this patent claim shall fall within the scope of the spirit of the present invention. .

100: Video recording system
110: user terminal
121~123: 1st to 3rd artificial intelligence cameras
CLST: cluster
131~13n: 1st to nth normal cameras
140: Video management server
250: main processor
270: Artificial intelligence processor

Claims (17)

In a method of operating any one of artificial intelligence cameras that can communicate with each other through a network:
Each of the artificial intelligence cameras includes an artificial intelligence processor configured to perform image analysis using a deep learning method,
The operation method is,
exchanging registration information with other artificial intelligence cameras through the network, thereby configuring a cluster for image analysis with the other artificial intelligence cameras;
sharing registration information of the normal cameras with the other artificial intelligence cameras so that the normal cameras can be accessed through the network;
When one of the normal cameras is assigned, accessing the assigned normal camera and receiving an image captured by the assigned normal camera;
Analyzing the received image using the artificial intelligence processor; and
An operating method comprising transmitting result information according to the analysis of the received video to an external device through the network along with the identifier of the assigned normal camera. delete delete delete delete delete delete delete delete delete delete delete optical system;
a first processor configured to process an image generated according to light received through the optical system;
a second processor that operates in response to control of the first processor and is configured to analyze the processed image based on a deep learning method; and
Includes a network interface that provides communication over a network,
The first processor,
By exchanging registration information with an external artificial intelligence camera through the network, a cluster for video analysis is formed with the external artificial intelligence camera,
Share the registration information of the normal cameras with the external artificial intelligence camera so that the normal cameras can be accessed through the network,
When one of the normal cameras is assigned, access the assigned normal camera and receive an image captured by the assigned normal camera,
Commanding the second processor to analyze the received image,
A camera configured to transmit result information according to the analysis of the received image to an external device through the network along with an identifier of the assigned normal camera. delete delete delete delete

Images