WO2020085558A1 - High-speed analysis image processing apparatus and driving method for apparatus - Google Patents

Abstract

The present invention relates to a high-speed analysis image processing apparatus and a driving method for the apparatus, and a high-speed analysis image processing apparatus according to an embodiment of the present invention can comprise: a communication interface unit for receiving a video image; and a control unit, which extracts an object to be analyzed from the received video image and analyzes same so as to generate attribute information of the object, and analyzes the video image on the basis of the generated attribute information of the object so as to generate the analysis result as metadata.

Description

High-speed analysis image processing device and driving method of the device

The present invention relates to a high-speed analysis image processing apparatus and a method of driving the apparatus, and more specifically, for video images of various formats, for example, based on a person's face or event option information related to a person, image analysis and search can be quickly performed. It relates to a high-speed analysis image processing apparatus to be processed and a driving method of the apparatus.

As the lives of modern people become more and more complicated and diversified, incidents and accidents are also increasing. As part of this reason, to monitor public places, CCTV cameras capture public places and automatically analyze images to extract a large number of unspecified objects and analyze motion to automatically alert the administrator when abnormal motion is detected, or There is an increasing demand for intelligent video surveillance systems that deliver information to other connected automation systems.

In recent years, a number of facilities have been installed for devices and methods for detecting specific events through images or videos acquired through surveillance CCTVs. Existing devices detect specific events that may occur depending on specific regions (eg : It is common to extract the tracking object from the image for the invasion of the center line, speeding, etc., and detect the occurrence of an event by the set value that the extracted object performs a certain action.

However, in the conventional object tracking method, object error detection occurs due to noise motion caused by various environmental factors of the object, and frequent object misclassification occurs due to performance limitations of the existing rule-based object classifier, In a complex environment where overlap between objects in a video is large and frequently occurs, there is a problem that it is difficult to properly detect the area of an individual object. In addition, there is a problem that the object detection method through comparison with the background model cannot be applied because the background changes frequently in the moving camera image.

An embodiment of the present invention provides a high-speed analysis image processing apparatus and a method for driving the apparatus for quickly processing image analysis and search based on, for example, human face or event option information related to a person, for video images of various formats It has a purpose.

The high-speed analysis image processing apparatus according to an embodiment of the present invention generates a communication interface unit that receives a video image, and extracts and analyzes an object to be analyzed from the received video image to generate attribute information of the object, and And a control unit that analyzes the video image based on attribute information and generates an analysis result as metadata.

The communication interface unit interlocks with an external device that performs object tracking-based image processing, and the control unit may perform high-speed analysis image processing based on a designated object using the received video image at the request of the external device.

The controller analyzes an event related to the object to be analyzed to further generate event information, and may include and store the generated event information in the metadata.

The controller may further generate deep learning-based metadata using attribute information of the object and the metadata.

The control unit may include a video processing unit that processes video images of different formats received through the communication interface unit.

The controller may search for and provide the generated metadata matching the search command based on a scenario-based search command received through the communication interface unit.

The communication interface unit may selectively receive video images of a photographing device, a removable storage medium (USB), and a third party device at a designated place.

In addition, a method of driving a high-speed analysis image processing apparatus according to an embodiment of the present invention is a method of driving a high-speed analysis image processing apparatus including a communication interface unit and a control unit, receiving a video image from the communication interface unit, and The control unit extracts and analyzes an object to be analyzed from the received video image to generate object attribute information, and analyzes the video image based on the generated object attribute information to generate an analysis result as metadata. do.

The communication interface unit interlocks with an external device that performs object tracking-based image processing, and the method of driving the high-speed analysis image processing device uses the received video image at the request of the external device to perform high-speed analysis image centering on a specified object. It may further include the step of performing the processing.

The driving method of the high-speed analysis image processing apparatus may further include analyzing an event related to the object to be analyzed, generating event information, and storing the generated event information in the metadata.

The driving method of the high-speed analysis image processing apparatus may further include generating deep learning-based metadata using attribute information of the object and the metadata.

The method for driving the high-speed analysis image processing apparatus may further include processing video images of different formats received through the communication interface unit.

The driving method of the high-speed analysis image processing apparatus may further include searching and providing the generated metadata matching the search command based on a scenario-based search command received through the communication interface unit.

The method for driving the high-speed analysis image processing apparatus may further include selectively receiving video images of a photographing apparatus, a removable storage medium, and a third party apparatus at a designated place.

According to an embodiment of the present invention, it is possible to increase accuracy in the replacement of incidents and accidents by quickly receiving and analyzing video images of different formats provided by various routes, that is, forensic images.

In addition, it extracts and analyzes an object to be analyzed from a video image that is not an object tracking method, generates attribute information of an object, stores metadata, and then provides metadata and a video image matching it based on the search word based on the attribute. Searches can be done quickly.

Furthermore, when performing a search based on the properties of an object, it is possible to add more event information related to a person, thereby further increasing the speed of the search. According to an embodiment of the present invention, since the event option information is further added during video analysis and data is stored, the search speed is reduced as the search amount decreases according to the addition of a search category (for example, events such as loitering, stoppage, vehicle stoppage, etc.). Will increase.

In addition, it is possible to secure the diversity of the search and increase the speed of the search by allowing the search to be performed even through a search word based on a scenario (eg, a sentence).

A search window (eg, UI) for inputting a person or event related to a person or a scenario-based search term may be formed simply and intuitively to facilitate user search convenience.

In addition, deep learning may be performed using the properties of the object, person attribute information, event information, etc., stored as metadata to increase the accuracy of the search, and additionally generate additional information to flexibly cope with events and accidents.

1 is a view showing a high-speed analysis video service system according to an embodiment of the present invention,

Figure 2 is an exemplary view schematically showing Figure 1,

3 is a block diagram illustrating the structure of the first image processing apparatus of FIG. 2,

4 is a block diagram illustrating the structure of the second image processing apparatus of FIG. 2,

5 is a block diagram illustrating another structure of the second image processing apparatus of FIG. 2,

6 is a block diagram illustrating the detailed structure of the image high-speed analysis unit of FIG. 5,

7 is a view showing a high-speed analysis video service process according to an embodiment of the present invention,

8 is a view for explaining an operation process between a forensic manager and a search client constituting the first image processing apparatus of FIG. 2;

9 is a view for explaining an operation process of the first image processing apparatus and the third party apparatus of FIG. 2,

10 is a diagram illustrating a search main screen,

11 is a diagram illustrating an FRS setup process,

12 to 20 are views for explaining an offline analysis screen,

21 to 30 are views for explaining an offline search screen, and

31 is a flowchart illustrating an operation process of a high-speed analysis image processing apparatus according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1 is a diagram illustrating a high-speed analysis image service system according to an embodiment of the present invention, and FIG. 2 is an exemplary diagram schematically showing FIG. 1.

1 and 2, the high-speed analysis image service system 90 according to an embodiment of the present invention includes a user device 100, a communication network 110, an image service device 120, and a third party device 130. ).

Here, "including some or all" means that some components such as the user device 100 or the third party device 130 are omitted, such that the video service system 90 is configured or the video service device 120 is configured. It means that the element can be integrated and configured in a network device (for example, a switching device, etc.) in the communication network 110, and is described as including everything in order to help a sufficient understanding of the invention.

The user device 100 is installed in a designated place and includes an imaging device such as CCTV for monitoring events and accidents, a desktop computer owned by users, a laptop computer, a mobile phone (eg, a smart phone), a tablet PC, and a smart TV. , A removable storage medium (eg, USB) 101 may be further included. Of course, such a removable storage medium may include a memory provided in a black box of a vehicle. The removable storage medium 101 may be directly connected to the control computer constituting the video service device 120.

The user device 100 may store an image captured through a camera (including a temporary storage site), and provide a captured image to the image service device 120 to request image analysis. Of course, if the video service device 120 acts as a control device, and the user device 100 is a CCTV-like recording device, the captured video is provided to the video service device 120 in real time or periodically to control it through analysis. Could be done.

The communication network 110 includes both wired and wireless communication networks. For example, a wired / wireless Internet network may be used or interlocked as the communication network 110. Here, the wired network includes an internet network such as a cable network or a public telephone network (PSTN), and the wireless communication networks include CDMA, WCDMA, GSM, Evolved Packet Core (EPC), Long Term Evolution (LTE), Wibro network, etc. It includes meaning. Of course, the communication network 110 according to an embodiment of the present invention is not limited thereto, and may be used as a cloud computing network under a cloud computing environment, a 5G network, etc. as a connection network of a next-generation mobile communication system to be implemented in the future. For example, when the communication network 110 is a wired communication network, an access point in the communication network 110 can access a telephone exchange or the like, but in the case of a wireless communication network, data can be accessed by accessing a SGSN or a Gateway GPRS Support Node (GGSN) operated by a communication company. It can process or connect to various repeaters such as BTS (Base Station Transmission), NodeB, and e-NodeB to process data.

The communication network 110 may include an access point. Access points include small base stations, such as femto or pico base stations, which are often installed in buildings. Here, the femto or pico base station is classified according to the maximum number of user devices 100 that can be accessed according to the classification of the small base station. Of course, the access point includes a user equipment 100 and a short-range communication module for performing short-range communication such as Zigbee and Wi-Fi. The access point can use TCP / IP or RTSP (Real-Time Streaming Protocol) for wireless communication. Here, short-range communication may be performed in various standards such as radio frequency (RF) and ultra-wideband communication (UWB), such as Bluetooth, Zigbee, infrared (IrDA), ultra high frequency (UHF), and very high frequency (VHF), in addition to Wi-Fi. You can. Accordingly, the access point can extract the location of the data packet, designate the best communication path for the extracted location, and transfer the data packet to the next device, for example, the video service device 120 along the designated communication path. An access point can share multiple lines in a typical network environment, including routers, repeaters and repeaters, for example.

The video service device 120 may serve as a control device for monitoring a corresponding area through a user device 100 installed in a designated area, for example, a captured image provided by CCTV. Of course, it will not be particularly limited. For example, according to an embodiment of the present invention, a server of a company for providing a service by performing a high-speed analysis operation of an image may be included. The video service device 120 includes a DB 120a for storing a large amount of video data, and may further include a server and a control computer.

In addition, the video service device 120 may be constructed in a variety of forms. For example, it can operate in the form of a single server, and multiple servers can work together. For example, the image service device 120 may include, for example, a first image processing device 121 and a second image processing device 123 as shown in FIG. 2. Through this, the image service device 120 of FIG. 1 can rapidly increase the image processing speed by collaborating or distributing the image processing operation.

According to an embodiment of the present invention, if the first image processing apparatus 121 performs an object tracking-based image processing operation, the second image processing apparatus 123 performs an object image-based image processing operation. It can be considered that object tracking is a method of predicting the motion of the extracted object by extracting the object from the first unit image of the image. Motion tracking is usually done in the form of a vector, i.e. direction and distance calculation. On the other hand, the object image-based image processing extracts a designated object (eg, a person, a vehicle, etc.) to be analyzed for each unit image (for example, the first to Nth unit images), and extracts for each unit image. It is a method of comparing the property information of an object and determining whether it is the same object. Therefore, in the case of a person, if there is a feature point on the face, the feature point information becomes attribute information. Also, in the case of a vehicle, shape or color may be attribute information. At this time, the final determination of the object may be made through deep learning. For example, similar objects are classified as candidate groups, and the properties of the object are finally determined through deep learning.

As described above, the second image processing apparatus 123 determines an object based on the attribute information and analyzes a correlation with surrounding objects or objects. Through this, an event is derived. Then, part or all of the attribute information, the event information, and the captured image matching it are generated as metadata. Of course, in the process of deriving the event, the second image processing device 123 may refer to a predetermined rule (or policy). In addition, the second image processing apparatus 123 may further generate new information by performing a deep learning operation using the stored metadata, and may also generate prediction information in the process.

Meta data for a specific video image generated in this way facilitates a search. Assume that the first image processing device 121 is a control device, for example, a video analysis request from a third party device 130 such as a police station or a government office. In this case, the first image processing device 121 may perform image analysis based on object tracking on its own, or may request the second image processing device 123 to perform object image based image analysis. . And, you can check the result of the video analysis. For example, this may be possible by storing data in DB 120a and then sharing it.

The search will be described in detail later, but first, the first image processing apparatus 121 and the second image processing apparatus 123 have different search categories, and search results are very different. In other words, the second image processing device 123 may further search the image analysis result using attribute information (eg, a face), and detect only a designated event (eg, as the first image processing device 121). Rather than doing so, it can be extended to further search by adding various event options. Here, the event option information may include roaming, stopping, vehicle stopping, and the like. Therefore, the speed of the search is fast and the search term is added, so the accuracy of the search is increased.

First of all, the first image processing device 121 may provide a scenario-based search word to search for a sentence, not a word, that is, a natural language, and increase the accuracy of information by a deep learning operation and receive prediction information together. For example, suppose the video image is a crime image. Since video can be secured through various channels based on the person, it is possible to provide prediction results by performing accurate prediction based on this. Scenario-based search terms may have a series of events such as, for example, 'black among illegally turning vehicles, a taxi' or 'a person wearing a hat running with a bag'.

In the embodiment of the present invention, the term i-forensic is used in that it processes video images of different formats obtained through various paths. This can be viewed as implying that, for example, picture-based image analysis is performed, but analysis and search can be performed in a short time by receiving images recorded in various formats. Video image means a video signal. Generally, the video signal is divided into a video signal and an audio signal, and further includes additional information. Of course, the additional information includes various information such as date and time information. In the embodiment of the present invention, the term video image is mainly used, but it may not mean only a video signal. Therefore, video images may be used interchangeably with video data and image data.

Incidentally, an image processing operation such as image analysis based on an object image performed by the second image processing apparatus 123 analyzes pixel values in a picture (or macroblock) image to classify object boundaries, and also classifies them. It is possible to analyze what type of object is by analyzing the pixel value of the object. For example, a person has a black head, a face is flesh-colored, and the two parts are connected to each other. Therefore, it is possible to classify human objects by extracting black and flesh-colored areas, and analyze corresponding parts to analyze feature points. Since there are various types of people or human faces, it may be possible to classify based on data stored in the form of a template. Therefore, if the characteristic point of a specific person is found, it can be judged as the same person even if there is a personal change, such as wearing the glasses.

For example, the shape of the nose, the shape of the ear, or the shape of the jaw line may be a characteristic point of a person, and on the basis of this, it is determined whether the same person as the subsequent unit images. In this way, an event is derived through correlation between the detected person and surrounding objects or objects. For example, a specific person was found in the unit video, and it was analyzed that the person is holding garbage. Then, in the Nth video, if the garbage is removed from the hand, and it is found in the dump site in the video, it is judged that the person dumped the garbage in an inappropriate place. Since such event generation can be analyzed in various forms, the embodiment of the present invention will not be limited to any one form.

On the other hand, in the embodiment of the present invention, the first image processing apparatus 121 and the second image processing apparatus 123 are interlocked with the image service apparatus 120, but this is configured in the form of a single server, for example. It may be formed in a form constituting the first module and the second module. Here, if the first module performs object tracking based image processing, the second module may perform object image (or photo) based image processing. Regarding the operation, it has already been described in detail above, and thus, further explanation is omitted. In addition, the image service device 120 may further interwork a third image processing device for high-speed analysis and search operations. For example, the third image processing apparatus provides an analysis result by performing an analysis operation based on an object image only for vehicle analysis on the received video image. Since various types of system design are possible, the embodiment of the present invention will not be limited to any one form.

The third party device 130 includes a server operated by a government office such as a police station, and a server of a company providing other content videos. For example, a control device operated by a local government may be a third party device 130. Of course, such a control device may be preferably the video service device 120 according to an embodiment of the present invention. However, it will not be particularly limited. Since the third party device 130 can be used for various purposes through image analysis, it should be understood as a provider that provides content, that is, a video image.

3 is a block diagram illustrating the structure of the first image processing apparatus of FIG. 2.

3, the first image processing apparatus 121 according to an embodiment of the present invention includes a communication interface unit 300, a control unit 310, a forensic image execution unit 320, and a storage unit 330. Includes some or all.

Here, "including some or all" is configured by omitting some components, such as the communication interface 300 or the storage unit 330, or some components such as the forensic image execution unit 320, the control unit ( 310) means to be integrated with other components such as, and is described as including all in order to help a sufficient understanding of the invention.

For example, when the forensic image execution unit 320 is integrated with the control unit 310, it may be referred to as a 'forensic image processing unit'. The forensic image processing unit may execute one software to perform a control operation and a forensic image processing operation together. Alternatively, the forensic image processing unit may be configured to operate in hardware, software, or a combination thereof. Furthermore, the control unit 310 may include a CPU and a memory. When configured in the form of an IC chip, a program for forensic image processing is stored in a memory and the CPU executes it. The processing speed increases rapidly.

Further, as described above, the forensic image execution unit 320 may perform image analysis based on an object image, but may also perform an image processing operation based on object tracking. For example, the former may be executed in the first module, and the latter may be executed in the second module.

As described above, the first image processing apparatus 121 according to the embodiment of the present invention may have a different configuration depending on how the system is configured. Above all, it is clear to perform image analysis based on object images and store the results in the form of metadata to provide search results quickly when a search request is made. Therefore, the embodiment of the present invention will not be particularly limited to any one form.

The communication interface 300 may communicate with the user device 100 and the sub-party device 130 of FIG. 1, respectively. To this end, a communication module may be included. Since the communication interface unit 300 processes video images, operations such as modulation and demodulation, encoding and decoding, muxing, and demuxing may be performed, but this may also be performed by the controller 310.

The communication interface 300 transmits this to the control unit 310 when, for example, a third party device 130 requests analysis of a video image, or when an image is provided from a photographing device such as a CCTV as the user device 100. You can.

In addition, the communication interface 300 may perform a setting operation for performing a forensic operation with the second image processing device 123 at the request of the control unit 310, through which the second image processing device 123 As a result, analysis results can be provided by requesting image processing.

The control unit 310 is responsible for the overall control operation of the communication interface unit 300, the forensic image execution unit 320, and the storage unit 330 constituting the first image processing unit 121. For example, the control unit 310 may control the forensic image execution unit 320 to perform an operation for performing a forensic operation with the second image processing apparatus 123. In other words, after the first image processing device 121 and the second image processing device 123 perform a setting process for interlocking the image analysis operation, for example, the control unit 310 performs a forensic image execution unit. According to the request of 320, the second image processing apparatus 123 may request an image analysis for the video image, and then provide a search term to receive the searched result.

For example, the control unit 310 may search for a video image by selecting a person's face or a person category as a search term according to the operation of the forensic image execution unit 320, and search for scenario-based search terms or event option information. Based on this, various types of searches can be additionally performed. In other words, the first image processing device 121 may analyze the A range and search, and the second image processing device 123 may analyze and search the B range, and the first image processing device through the forensic operation ( If the analysis and search is requested from the 121) to the second image processing apparatus 123, the first image processing apparatus 121 operates to view the analysis or search results of the second image processing apparatus 123 in the B range. will be. It can be seen that the control unit 310 and the forensic image execution unit 320 perform this.

As described above, the forensic image execution unit 320 executes an interlocking program to allow the first image processing unit 121 to view the analysis results of the video image analysis of the second image processing unit 123. This may include various UX / UI programs. For example, when a user sets a forensic operation through a UI window or when a user provides a search term with a property of a person, such as a 'face' category, the second image processing device 123 ) To provide various search results.

The storage unit 330 may store various data processed by the first image processing device 121 and may temporarily store the data. For example, when the first image processing device 121 is interlocked with the DB 120a, temporary data may be stored in the storage unit 330 and permanent data may be stored in the DB 120a. The data stored in the storage unit 330 is output when requested by the control unit 310.

4 is a block diagram illustrating the structure of the second image processing apparatus of FIG. 2.

4, the second image processing apparatus 123 according to an embodiment of the present invention includes some or all of the communication interface 400 and the image high-speed processing unit 410, where "some or all "Including" is the same as the above.

The communication interface 400 may communicate with the first image processing apparatus 121 of FIG. 2 according to an embodiment of the present invention. When the first image processing device 121 provides a video image and requests analysis, the video image and the analysis request are transmitted to the image high-speed processing unit 410.

In addition, the communication interface unit 400 analyzes the video image, such as person attribute information, event information, correlation information, and the video image matching the information in the form of metadata, the image high-speed processing unit 410 It can be stored in the DB (120a) on request.

Then, the communication interface unit 400 provides a search term, such as an attribute-based search term, a scenario-based search term, and an event option-based search word from the first image processing device 121 to the image fast processing unit 410 Search results are provided accordingly.

The image high-speed processing unit 410 performs an object image-based analysis operation when an analysis request is made for the received video image. In this regard, the above description has been sufficiently described, and further explanation is omitted.

In addition, the image high-speed processing unit 410 may store the analysis results in the DB 120a so that the first image processing device 121 accesses the DB 120a to perform the above search. That is, the search can be performed by the first image processing device 121 directly accessing the DB 120a, but various methods are possible such as receiving the search results indirectly via the image high-speed processing unit 410, which is a system. Since it may vary according to the designer's intention, the embodiment of the present invention will not be particularly limited to any one form. However, the former would be desirable in view of the speed of data processing.

5 is a block diagram illustrating another structure of the second image processing apparatus of FIG. 2, and FIG. 6 is a block diagram illustrating a detailed structure of the high-speed analysis image execution unit of FIG.

As shown in FIG. 5, the second image processing apparatus 123 'according to another embodiment of the present invention includes a communication interface unit 500, a control unit 510, an image high-speed analysis unit 520, and a storage unit 530. ).

Here, "including some or all" means that some components, such as the storage unit 530, are omitted, so that the second image processing apparatus 123 'is configured or some components, such as the image high-speed analysis unit 520, are included. It means that it can be integrated with other components, such as the control unit 510, etc., and is described as including everything in order to help the understanding of the invention.

Compared to the second image processing apparatus 123 of FIG. 4, it is considered that the second image processing apparatus 123 'of FIG. 5 is that the control operation and the high-speed analysis (and search) operation of the image are dualized and processed. You can. To this end, they may be separated by hardware, software, or a combination thereof to perform different operations. That is, it can be seen that the control unit 510 performs only the control operation, and the high-speed analysis operation of the image is performed by the image high-speed analysis unit 520. Of course, the image high-speed analysis unit 520 performs object image-based analysis. For example, it can be considered as a method of analyzing attributes by gapping objects in the form of images and analyzing pixel values for objects in the captured images.

Except for the above points, the communication interface unit 500, the control unit 510, the image high-speed analysis unit 520, and the storage unit 530 of FIG. 5 communicate with the image service device 120 or FIG. 4 of FIG. Since the contents of the interface unit 400 and the image high-speed processing unit 410 are not significantly different, the contents will be replaced.

Meanwhile, the image high-speed analysis unit 520 of FIG. 5 may have the same structure as the image high-speed analysis unit 520 'of FIG. 6. The image high-speed analysis unit 520 of FIG. 5 is a part or all of the video processing unit 600, the video search unit 610, the scheduling unit 620, the manual processing unit 630, and the bookmark processing unit 640 as shown in FIG. It may include. Here, "including some or all" has the same meaning as above.

The image high-speed analysis unit 520 of FIG. 5 may further perform various operations in addition to the high-speed analysis described above. The video processing unit 600 may process video data provided in various formats by processing data in a format of a corresponding image or converting data into a specified format, and converting the data into a same format. In an embodiment of the present invention, video data provided in various formats may be converted into a specified format, then analyzed and converted into the same format and then exported.

The video search unit 610 may allow an attribute-based search, for example, a search to be performed in a 'face' category, or a search based on event option information, and a scenario-based search. The scenario-based scenario is to provide a search result based on the analysis of a search word provided in the form of a sentence or, more precisely, a short sentence. In other words, it can be considered as similar to a keyword-based search, but there may be a difference in that keywords correspond to words and scenarios are sentences. For example, the scenario sentence may include a complex scenario such as "black of a vehicle that is illegally turning, in the case of a taxi" or "a person wearing a hat running with a bag".

The scheduling unit 620 performs a schedule management operation, and may be in charge of an operation in which video analysis is automatically registered and automatically generated at a specified time once or periodically (daily, weekly, monthly).

The manual processing unit 630 may perform a manual function, such as an operation of providing help. It is to provide the manual so that it can be used directly on i-Forensics.

The bookmark processing unit 640 may perform various operations related to bookmarking a favorite image, such as designating a bookmark (interest list), deleting a bookmark, exporting a bookmark list, managing multiple bookmarks, and controlling bookmark deletion (protection).

In addition, the image high-speed analysis unit 520 of FIG. 5 may further include a configuration for performing various operations as shown in [Table 1] and [Table 2]. That is, since FIG. 6 is only an example, the configuration may be further added to the configuration of FIG. 6 by SW modules, HW modules, and combinations based on the contents of [Table 1] and [Table 1]. 6 is a configuration in which only representative operations are described in the form of modules.

7 is a view showing a high-speed analysis video service process according to an embodiment of the present invention.

Referring to FIG. 7, the first image processing apparatus 121 of FIG. 2 may include a forensic manager 121a and a search client 121b. This may be in the form of a SW module, for example. The forensic manager 121a may be in charge of management or control of the first image processing device 121, and the search client 121b may perform a search-related operation. FIG. 7 shows operations between the manager 120a and the client 121b in the DB 120a, the third party device 130, and the first image processing device 121, and further, the second image processing device 123, for example. .

7, the first image processing device 121 requests and receives a video image from a DB 120a or a third party device 130 such as a VMS (Virtual Memory System), and then receives the second image processing device 123 Shows that the analysis of the video image is completed by inquiring or requesting with (S701 ~ S712). Here, steps S705 to S707 are a process of requesting an inquiry or analysis to the second image processing apparatus 123, steps S708 to S710 are processes for performing a high-speed analysis operation, and steps S711 and S712 are processes for completing the analysis. .

FIG. 8 is a diagram illustrating an operation process between a forensic manager and a search client constituting the first image processing apparatus of FIG. 2.

Referring to FIG. 8, the forensic manager 121a and the client 121b may specifically operate in a form of processing a video list as in FIG. 8 (S800 to S804). For example, when a user such as a control agent inputs a specific search word through the first image processing device 121, a list of various video images corresponding to the search word may be first provided to the user. Through this list, you can select the video you are looking for and receive and play the selected video. Alternatively, the analyzed result may be provided and displayed on the screen.

9 is a view for explaining an operation process of the first image processing apparatus and the third party apparatus of FIG. 2.

Referring to FIG. 9, the first image processing device 121 may request meta data stored in the DB 120a and receive it as a stream along with a video image to display it on the screen (S900 to S905). Such a process may receive meta data from the DB 120a through a collaborative operation between the manager 121a and the client 121b, for example, a video image according to a specific search term and various information matching it and display it on the screen.

10 is a diagram illustrating a search main screen, and FIG. 11 is a diagram illustrating an FRS setting process.

Referring to FIGS. 10 and 11 together with FIG. 2 for convenience of description, the first image processing apparatus 121 according to an embodiment of the present invention may include a control computer, a control monitor or a display board connected to the control computer. In order to enable an i-forensic operation according to an embodiment of the present invention to be performed on the control computer, an FRS connection establishment operation may be performed as shown in FIGS. 10 and 11. In the screen displayed when the environment setting button 1000 on the upper right in FIG. 10 is selected, when the FRS connection setting item 1010 is clicked as shown in FIG. 11, a specific setting screen pops up. Thereafter, an IP address of the second image processing device 123 may be input to the pop-up window to interlock with each other. At the bottom of the pop-up window, as shown in FIG. 11, status information 1100 indicating whether or not a connection is made may be displayed. Through this, the i-forensic interlocking process of the first image processing apparatus 121 and the second image processing apparatus 123 is completed.

12 to 20 are views for explaining an offline analysis screen.

For convenience of description, referring to FIGS. 12 to 20 together with FIG. 2, the first image processing device 121 of FIG. 2 clicks the file import button 1200 displayed on the screen as shown in FIG. 12 to search for a desired file. Can be loaded. At this time, the imported files are registered in the analysis channel list area 1210 on the left side of the screen, that is, the first area. In the case of a file that has already been analyzed during the file import process, the file is stored in the list area 1210, but analysis may not be possible.

Subsequently, as shown in FIG. 13, after selecting a specific video file in the analysis channel list area 1210 and selecting the offline analysis button 1300, the video image requesting analysis to the video display area, that is, the second area, may be played. . At the bottom of the second area in which the video image is reproduced, the image is reproduced small in the form of a thumbnail image, and additional information such as time may be displayed on the image.

In addition, when performing offline analysis, when a user requests to analyze a video image that is an analysis target 1410 in the analysis channel list area 1210 as shown in FIG. 14, a pop-up window 1400 for setting an analysis type is displayed. You can call it and set the analysis type. It can be seen that the 'face' category is added to the pop-up window 1400. This can be regarded as an item for checking attribute information analyzed based on a person's object image according to an embodiment of the present invention.

According to an embodiment of the present invention, the video image of the analysis target 1410 selected may be analyzed in three types per file. This is made according to the designated method according to the embodiment of the present invention, and may be changed as much as possible. However, since the embodiment of the present invention provides data for video images of various formats, it is desirable to support more than that. In addition, it can be designed so that re-analysis is not possible for the analysis target or the analysis result value can be maintained as long as there is no request for deletion.

When analysis of the selected video images is completed through the process as shown in FIGS. 14 and 15, the video images disappear from the analysis channel list area, and the analyzed video images are analyzed in the standby / progress list area 1600 as shown in FIG. ). The area may be a third area. The third area may be divided into a small area, an area including an analysis standby / progress list, an area including an analysis completion list, and an area including an analysis failure list.

At this time, various video images contained in the third area may be brought back to the analysis channel list area as shown in FIG. 17 to perform analysis again. The third area is divided into analysis completion when there is an object having a desired analysis result as shown in FIG. 18, and when analysis is completed but there is no object, that is, no object, analysis failure list cannot be completed due to network errors or file problems. Will be included in the domain.

In the case of a video image in which the analysis has failed as shown in FIG. 18, the analysis may be retried under different conditions as in FIG. For example, if the analysis fails as a result of an attempt to search with a general object item as shown in FIG. 19, the analysis is retried as shown in FIG. 20 based on the attribute information of the person according to the embodiment of the present invention.

As a result, you will receive the results according to the person search.

21 to 30 are views for explaining an offline search screen.

For convenience of description, referring to FIGS. 21 to 30 together with FIG. 2, the first image processing device 121 of FIG. 2 according to an embodiment of the present invention may display a search screen as shown in FIG. 21 on a monitor screen. That is, the search item 2100 is selected on the main screen. Subsequently, by selecting the video list item 2110 to be searched, the searched list may be retrieved as shown in FIG. 22. This is provided in the list of completed analyzes. Of course, the user can delete specific items on the list.

After putting the video images of the list in the video list area to search for, that is, the fourth area, when a specific video image is not selected, the video reproduced on the screen does not appear as shown in FIG. 23.

24, the user selects a specific video image and sets various search expressions. In this case, in the case of video images of different formats, search conditions (expression) customized to the video image are displayed on the screen. For example, the search window 2500 for the first format video image and the second format video image may display different items.

When playing a video image, since a playback option such as continuous playback of an object section is added in addition to normal playback as shown in FIG. 25, a playback method may be determined by selecting it.

In addition, when setting search conditions, a search type item in the search window 2500 may be selected to search for people, vehicles, and other unidentified objects constituting a general object. You can perform a search.

After the search is performed, when a video is played on the screen, a play period can be designated through a play bar as shown in FIG. 27, and a play time can also be set. If the object section is continuous playback as shown in FIG. 28, continuous playback is performed.

FIG. 29 shows that multiple selection of thumbnails and continuous playback thereof can be performed, and FIG. 30 shows that a specific video image, such as a clip image, can be set as a favorite by selecting the favorite button 3000 have.

31 is a flowchart illustrating an operation process of a high-speed analysis image processing apparatus according to an embodiment of the present invention.

For convenience of description, referring to FIG. 31 together with FIGS. 1 and 2, as a high-speed analysis image processing apparatus, an image service apparatus 120 or a first image processing apparatus 121 according to an embodiment of the present invention (hereinafter, a first image) A processing device) receives a video image (S3100). At this time, the received video image includes images of different formats.

Subsequently, the first image processing device 121 extracts and analyzes the face image of the person from the received video image to generate face attribute information, and analyzes the video image based on the generated face attribute information to analyze the result as metadata. It is created (S3110).

After the meta data stored as the result of the analysis is stored in the DB 120a as shown in FIG. 1, the first image processing device 121 provides video analysis results according to various search expressions of the user. First of all, in the embodiment of the present invention, character-oriented analysis was further performed, and of course, the analysis is performed by a method of analyzing an object image in an image, but further analyzes a video image analysis result based on the attribute information of the corresponding person. Will be provided.

At this time, the search may be performed by adding event option information of the person, and above all, a scenario-based search may be performed.

Since these contents have been sufficiently explained above, further explanation is omitted.

On the other hand, that all components constituting the embodiments of the present invention are described as being combined or operated as one, the present invention is not necessarily limited to these embodiments. That is, if it is within the scope of the present invention, all of the components may be selectively combined and operated. Further, although all of the components may be implemented by one independent hardware, a part or all of the components are selectively combined to perform a part or all of functions combined in one or a plurality of hardware. It may be implemented as a computer program having a. The codes and code segments constituting the computer program may be easily deduced by those skilled in the art of the present invention. Such a computer program is stored in a computer-readable non-transitory computer readable media, and read and executed by a computer, thereby implementing an embodiment of the present invention.

Here, the non-transitory readable recording medium means a medium that stores data semi-permanently and that can be read by a device, rather than a medium that stores data for a short time, such as registers, caches, and memory. . Specifically, the above-described programs may be stored and provided on a non-transitory readable recording medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

Although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and it is usually in the technical field to which the present invention pertains without departing from the gist of the present invention as claimed in the claims. It is of course possible to perform various modifications by a person having knowledge of, and these modifications should not be individually understood from the technical idea or prospect of the present invention.

** Explanation of codes

100: user device 101: removable storage medium

110: communication network 120: video service device

121: first image processing device 123, 123 ': second image processing device

130: third party device 300, 400, 500: communication interface unit

310, 510: control unit 320: forensic image execution unit

330, 530: storage unit 410: image high-speed processing unit

520, 520 ': image high-speed analysis unit 600: video processing unit

610: video search unit 620: scheduling unit

630: manual processing unit 640: bookmark processing unit

Claims (14)

1. A communication interface unit that receives a video image; And

   A control unit for extracting and analyzing an object to be analyzed from the received video image to generate attribute information of the object, and analyzing the video image based on the attribute information of the generated object to generate an analysis result as metadata;

   High-speed analysis image processing device including.
2. According to claim 1,

   The communication interface unit interlocks with an external device that performs object tracking-based image processing, and the controller performs high-speed analysis image processing based on a specified object using the received video image at the request of the external device. Processing device.
3. According to claim 1,

   The controller analyzes an event related to the object to be analyzed, further generates event information, and stores the generated event information in the metadata to store a high-speed analysis image processing apparatus.
4. According to claim 1,

   The controller is a high-speed analysis image processing apparatus that further generates deep learning-based metadata using the attribute information of the object and the metadata.
5. According to claim 1,

   The control unit includes a video processing unit that processes video images of different formats received through the communication interface unit.
6. According to claim 1,

   The control unit searches for and provides the generated metadata matching the search command based on a scenario-based search command received through the communication interface unit.
7. According to claim 1,

   The communication interface unit is a high-speed analysis image processing device for selectively receiving the video image of a photographing device, a removable storage medium (USB) and a third party device at a designated place.
8. A method of driving a high-speed analysis image processing apparatus including a communication interface and a control unit,

   Receiving a video image from the communication interface; And

   The control unit extracts and analyzes an object to be analyzed from the received video image to generate attribute information of the object, and analyzes the video image based on the generated attribute information to generate an analysis result as metadata. ;

   Method of driving a high-speed analysis image processing apparatus comprising.
9. The method of claim 8,

   The communication interface unit interlocks with an external device that performs object tracking-based image processing, and performs high-speed analysis image processing based on a designated object using the received video image when requested by the external device. Driving method of high-speed analysis image processing device.
10. The method of claim 8,

    Generating event information by analyzing an event related to the object to be analyzed; And

    And storing the generated event information by including the metadata in the meta data.
11. The method of claim 8,

    And generating deep learning based meta data using the attribute information of the object and the meta data.
12. The method of claim 8,

    And processing video images of different formats received through the communication interface unit.
13. The method of claim 8,

    And a sp .. step of searching for and providing the generated metadata matching the search command based on the scenario-based search command received through the communication interface unit.
14. The method of claim 8,

    A method of driving a high-speed analysis image processing apparatus further comprising; selectively receiving a video image of a photographing apparatus, a removable storage medium, and a third-party apparatus at a designated place.

Images