KR102223010B1 - Apparatus and Method for Providing Multi frame Image display searching technology Having Object Identifying Function - Google Patents

Abstract

The present invention relates to a multi-screen search service providing device having an object recognition function and a method thereof, which simultaneously displays image data extracted by a search request including object information on at least one split screen by listing the image data in the order of photographing time. The device comprises: a search request receiving unit for receiving a screen search request including object information from a search requester; an object recognition unit for recognizing the object information included in the screen search request received by the search request receiving unit; an image data extraction unit for extracting image data including the object information recognized by the object recognition unit from image data photographed by a plurality of cameras; and a screen display unit for listing the image data extracted by the image data extraction unit in the order of photographing time and displaying the image data on at least one split screen. An image portion containing a specific object is extracted from the image data recorded by the plurality of CCTV cameras so as to quickly identify the movement of the specific object during search, thereby saving search time.

Description

Apparatus and Method for Providing Multi frame Image display searching technology Having Object Identifying Function}

The present invention relates to a system and method for supporting multi-screen search technology capable of re-adjusting and editing channels based on object recognition, and more specifically, object information in a situation in which a CCTV system storing image data including object information is constructed. The image data extracted by a specific search request can be displayed simultaneously in at least one split screen by setting the shooting time sequence or the priority of a specific parameter to be listed. At this time, the listed image data can be displayed on the multiple search screen at random by the operator. The present invention relates to a system and method for supporting a multi-screen search technology capable of re-adjusting and editing channels based on object recognition that can be deleted from a multi-search screen for newly combined or unnecessary video channels.

In general, there are two important functions in the important functions of a CCTV system. It is to prevent crime and secure evidence of crime in the event of a crime.

Recently, CCTV (Closed Circuit Television) has been installed inside and outside buildings as well as alleys and roads for crime prevention, traffic information collection, and monitoring purposes. After an accident occurs, the police or manager must check the surveillance image taken through CCTV while displaying it on the monitor in order to check the accident. At this time, if you do not know the exact time of occurrence, it takes a lot of time because you have to check the captured surveillance images on a monitor from start to finish.

The recent technological advancement in the CCTV field has advanced to the level where a 12M pixel camera can capture an image and store/search/network transmission of it to a recording device. In addition, through object recognition (person/car/motorcycle/bicycle, etc.) on a 5M pixel camera, the characteristics of the object (face part extraction, gender classification, age estimation, clothes color, beard presence, wearing glasses, wearing a mask, vehicle license plate, etc.) Extraction, vehicle manufacturer, vehicle color, vehicle type classification, people riding motorcycles, people riding bicycles, etc.) are analyzed in the video, and metadata is transmitted to the recording device along with the video data, and the metadata along with the video to the recording device. A technology for storing events and searching for events based on metadata is being used.

The conventional technology has evolved from a recorder to a fast search using metadata from a camera stored image of a specific channel. In this case, a search function capable of searching for an object specified as metadata in a thumbnail method may be provided.

However, you can only list a myriad of images of specific objects in the thumbnail method, and the images listed in the thumbnail method are at the level of playing the image on a single screen, and there is no function to display other images at the same time. There is a technical limitation that cannot clearly express the movement of a specific object because there is no sorting function that can combine or delete them.

The basis of the existing recorder search function is that there is a limit to simultaneously displaying images recorded at the same time. For example, while searching for 16-segmented search images at the same time, if vehicle A passes through the front door of the apartment, vehicle A is photographed at 12:00 on the first camera image. In addition, when vehicle A is photographed at 12:05 on camera 5, and vehicle A is photographed on camera 10 at 12:10. The existing method is a thumbnail method, and each of the three videos is listed so that they can be searched.

Therefore, the searcher has the hassle of having to check each video separately. When searching in the 16-segment mode, 15 channels are filled with unnecessary images while searching from 12: 00 to 12: 10, and there is an inconvenience of having to look at camera images 1, 5, and 10 while moving their eyes busy. .

In order to improve this, through the present invention, for example, in a state of 16 division, an image taken at 12:00 is placed on channel 1, and a video from camera 5 taken at 12:05 is placed on channel 2, and the third On the channel, the 10th camera video taken at 12:10 is placed, and the remaining unnecessary 2, 3, 4, 6, 7, 8, 9, 11, 12, 13, 14 are unnecessary. Camera images No., No. 15, and No. 16 are deleted, and the multi-search screen is readjusted into four divisions by 16 divisions, so that the operator can add or delete the multi-search screen so that it can be re-adjusted and rearranged. Through this, it is possible to minimize the search time during search and to minimize the time and capacity for backing up search images. In addition, in a situation in which the recent development of multi-channel recording apparatuses is expanding to 32/36/64/72/128 channels, the technology of the present invention is further required.

Meanwhile, Korean Patent Application Publication No. 10-2008-0082836 discloses a method and system for generating a video synopsis. The synopsis method is a method of creating a shortened video by combining selected parts from a plurality of original images. By compressing time through this synopsis method, it is possible to view a summary of a video clip within a short time.

In the case of the above technique, a portion representing the non-spatial overlapping appearance of an object is copied and displayed from at least three different input frames to at least two consecutive frames. Therefore, depending on the number of objects that appear, there are times when many objects are displayed at once, and there are times when fewer objects are displayed, so when many objects are displayed at once, monitoring is not easy, and when the number of displayed objects is small, monitoring time is unnecessary. It takes.

KR 10-2008-0082963 A KR 10-2009-0117771 A KR 10-2010-0026631 A

The present invention is derived from such a technical background, and object recognition-based channel readjustment editing is possible to save search time when searching by extracting an image portion containing a specific object from image data recorded by a plurality of CCTV cameras. Its purpose is to provide a system and method for supporting multi-screen search technology.

Specifically, unnecessary video channels can be deleted from the search screen when searching, and video channels of interest can be selected, and video channels of interest can be changed from the search channel to any location regardless of the existing recording channel number. We want to improve our search skills so that you can search.

In addition, since the image data on which the specific object to be searched is photographed can be divided according to the passage of time and checked on one screen, the video playback time can be efficiently adjusted, so that the fluctuations of the object that needs to be searched over time can be grasped at a glance. Provides a multi-screen search technology support system and method capable of re-adjusting and editing channels based on object recognition that can reduce the hassle and manpower of searching necessary to grasp the fluctuations over time, such as the movement path of a specific object. I want to.

The present invention for achieving the above object includes the following configurations.

That is, the multi-screen search technology support system according to an embodiment of the present invention receives image data captured from at least one camera, recognizes object information included in the received image data, and generates metadata, A storage unit that matches and stores the received image data and metadata generated by the object recognition unit, receives a screen search request including object information, and identifies and stores object information included in the received screen search request An image data extraction unit that extracts image data including the identified object information among image data stored in the unit, and the image data extracted from the image data extraction unit are arranged in the order of photographing time and displayed on at least one divided screen. Includes a screen display.

In addition, the system for supporting multi-screen search technology according to another embodiment of the present invention includes an object recognition unit configured to generate metadata by identifying object information included in the captured image data. An image data receiving unit for receiving data and metadata generated by the object recognition unit, a storage unit for matching and storing the image data and metadata received by the image data receiving unit together, and receiving a screen search request including object information, and , An image data extracting unit configured to extract image data including the identified object information among image data stored in the storage unit by grasping object information included in a received screen search request, and an image extracted from the image data extracting unit. And a screen display unit that arranges data in the order of photographing time and displays at least one divided screen.

On the other hand, the multi-screen search technology support method performed in a multi-screen search technology support system including an object recognition unit, a storage unit, an image data extraction unit, and a screen display unit, the object recognition unit Generating metadata by recognizing included object information, when an image data extraction unit receives a screen search request including object information, captures object information included in the received screen search request with the at least one camera Extracting image data including the identified object information from among image data stored in a storage unit that matches and stores the image data generated by the object recognition unit and metadata generated by the object recognition unit, and the image data extraction unit by a screen display unit And displaying the image data extracted from the image data in the order of photographing time in at least one divided screen.

According to the present invention, an object recognition-based channel readjustment editing that can reduce search time by extracting an image portion containing a specific object from image data recorded by a plurality of CCTV cameras and quickly grasping the movement of a specific object during search The effect of providing a multi-screen search technology support system and method is derived.

In addition, since multiple image data on which a specific object to be searched is photographed can be divided according to the passage of time and viewed on a single screen, the video playback time can be efficiently adjusted and the fluctuations of the object that need to be searched over time can be controlled. A multi-screen search technology support system and method that enables re-adjustment and editing of channels based on object recognition that can be identified at a glance, reducing the hassle and manpower of searching necessary to understand changes over time, such as the movement path of a specific object. Can provide.

1 is a block diagram showing the configuration of a multi-screen search service providing apparatus having an object recognition function according to an embodiment of the present invention;
2 is a block diagram showing the configuration of a multi-screen search service providing apparatus having an object recognition function according to another embodiment of the present invention;
3 is an exemplary view of a screen in which a search result is displayed on a screen display unit of a multi-screen search service providing apparatus according to an embodiment of the present invention;
4 is a flowchart illustrating a method of supporting a multi-screen search technology having an object recognition function according to an embodiment of the present invention.

It should be noted that the technical terms used in the present invention are only used to describe specific embodiments, and are not intended to limit the present invention. In addition, the technical terms used in the present invention should be interpreted as generally understood by those of ordinary skill in the technical field to which the present invention belongs, unless otherwise defined in the present invention, and is excessively comprehensive. It should not be construed as a human meaning or an excessively reduced meaning.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing the configuration of a system for providing a multi-screen search service having an object recognition function according to an embodiment of the present invention.

The multi-screen search technology support system having an object recognition function according to an embodiment of the present invention relates to a technology for effectively and quickly securing evidence after a crime occurs.

The multi-screen search technology support system 10 expresses search channels in multiple screens for efficient search and backup in a system including an object recognition camera or an integrated recording device (or distributed server, etc.) including an object recognition unit, and unnecessary search channels. This TEk can provide the convenience of searching that you can delete and add search channels you are interested in, and you can place and adjust the desired location among multiple screens.

In the multi-screen search technology support system 10 according to an embodiment of the present invention, only the image in which a specific object is photographed among images photographed by a plurality of cameras 20 in a multi-screen (two or more screens) search window. Channels can be sorted and displayed sequentially from No. 1.

For example, the image of camera 1, taken at 12:00, is expressed on channel 1, the image of camera 5, taken at 12:05, is expressed on channel 2, and the 10th, recorded at 12:10 When the camera image is expressed on camera 3, the searcher can follow the channels sequentially and easily check when and when the camera at which location took the picture of vehicle A, and can easily check the path of which vehicle A moved. will be.

For example, in the state of 16 division, a video shot at 12:00 is placed on channel 1, and a video from camera 5 shot at 12:05 is placed on channel 2, and recorded at 12:10 on channel 3 Arrange the video from the 10th camera, and the remaining unnecessary 2, 3, 4, 6, 7, 8, 9, 11, 12, 13, 14, 15, and 16 cameras The video is deleted and the multiple search screens are readjusted by 16 divisions into 4 divisions, so that the operator can add or delete multiple search screens so that they can be re-adjusted and rearranged. Through this, it is possible to minimize the search time during search and to minimize the time and capacity to back up search images.

Therefore, the multi-screen search technology support system 10 according to an embodiment of the present invention is very useful when searching for multiple screens at the same time, and in particular, unnecessary videos that do not contain a specific object to be searched are not displayed in the multi-screen search window. Therefore, it provides the efficiency of data search that can minimize the fatigue of the searching personnel and quickly collect the desired data.

In other words, it is possible to sequentially display images containing the object to be searched in the multi-image search window, and to back up only the necessary images, thereby shortening the backup time as well as the backup capacity by backing up unnecessary image data. This can be prevented from being wasted.

As shown in FIG. 1, the multi-screen search technology support system 10 according to an embodiment includes a communication unit 110, an image data receiving unit 120, an object recognition unit 130, a search request receiving unit 140, and an image data extracting unit. 150, a screen display unit 160, a backup unit 170, and a storage unit 180.

The multi-screen search technology support system 10 according to an embodiment includes a communication unit 110, an image data receiving unit 120, an object recognition unit 130, a search request receiving unit 140, an image data extracting unit 150, and a screen. A recording server, a network server, a search server, a backup server, which are implemented as a single physical device including all of the technical components such as the display unit 160, the backup unit 170, and the storage unit 180, or include some of them. It is possible to implement at least one or more physical devices such as an independent server for each function such as a display server. In addition, the object recognition unit 130 may be implemented in a form in which the communication unit 110 or the object recognition camera 20 and the user terminal 30 are integrated.

The communication unit 110 communicates with an internal element or at least one external terminal and a server through a wired/wireless communication network. In this case, the external arbitrary terminal may include a camera 20 and a user terminal 30.

Here, as the wireless Internet technology, wireless LAN (WLAN), DLNA (Digital Living Network Alliance), Wireless Broadband: Wibro, Wimax (World Interoperability for Microwave Access: Wimax), HSDPA (High Speed Downlink Packet Access) ), High Speed Uplink Packet Access (HSUPA), IEEE 802.16, Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), Wireless Mobile Broadband Service (WMBS), etc. In addition, the communication unit 110 transmits and receives data according to at least one wireless Internet technology in a range including Internet technologies not listed above.

In addition, short-range communication technologies include Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and Near Field Communication (NFC). , Ultra Sound Communication (USC), Visible Light Communication (VLC), Wi-Fi, Wi-Fi Direct, and the like may be included. In addition, wired communication technologies may include power line communication (PLC), USB communication, Ethernet, serial communication, optical/coaxial cables, and the like.

The communication method supported by the communication unit 110 is not limited, and a short-range wireless communication method between devices as well as a communication method using a communication network (for example, a mobile communication network, wired Internet, wireless Internet, broadcasting network) that the network may include. Data transmission and reception can be performed.

Here, the network is one of networks such as PAN (personal area network), LAN (local area network), CAN (campus area network), MAN (metropolitan area network), WAN (wide area network), BBN (broadband network), and the Internet. It may include one or more of any network.

In addition, the network may include any one or more of a network topology including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree, or a hierarchical network, but is not limited thereto. Does not.

The image data receiving unit 120 receives image data captured from the camera 20, the first CCTV, the second CCTV, ... through the communication unit 110, and performs a function of recognizing an object and generating metadata from the received image data. Carry out. In addition, the received image data is stored in the storage unit 180 together with the generated metadata. In addition, the camera 20 or the user terminal 30 may perform a function of recognizing an object and generating metadata.

The storage unit 180 is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg, SD or XD memory, etc.) , Magnetic memory, magnetic disk, optical disk, RAM (Random Access Memory: RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory: ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory) may include at least one storage medium.

In an embodiment, the storage unit 180 stores image data received by the image data receiving unit 120 and metadata about object information included in the image data together.

At this time, the image data receiving unit 120 analyzes one of face recognition, gender, age, clothes color, beard, glasses, mask, vehicle license plate, vehicle manufacturer, vehicle color, vehicle type, motorcycle occupant, and bicycle occupant characteristics. Based on the object information, the identified object information is stored together in the storage unit 180. Metadata is not limited thereto, and is interpreted to include all types of information other than the characteristics or propensity of the object.

The search request receiving unit 140 receives a screen search request including object information from a search requester. In one embodiment, the search request receiving unit 140 may receive a screen search request through a user interface provided in a terminal on which the search request receiving unit 140 of the multi-screen search technology support system 10 is mounted.

In another embodiment, the search request receiving unit 140 may receive a screen search request from the user terminal 30 through the communication unit 110 performing network communication.

In one embodiment, the screen search request includes object information. The search requester selects a channel of interest from a mouse, electronic pen, touch screen, motion (movement) sensor input, voice recognition, various sensor inputs, etc. from the simultaneous search screen window of two or more channels of multiple search screens provided by the screen display unit 160. I can. In addition, the search requester selects a specific person or object as an object of interest from the image data of the selected channel of interest.

To select an object of interest, the search requestor may directly input the object's metadata value or may use a photographic image to input the object of interest.

Then, the multi-screen search technology support system 10 according to an embodiment checks the metadata of the object of interest selected by the search requestor. In addition, metadata useful for search among the metadata of the object of interest can be selected as a search criterion.

Thereafter, the search requestor may request a search for stored image data by designating a search date and time range.

Accordingly, information on a specific person or vehicle is identified as metadata, and image data including a specific person or vehicle photographed in different time periods is provided as a search result. It becomes possible to grasp at a glance.

The user terminal 30 is implemented as a terminal device possessed or manipulated by a search requestor. For example, the user terminal 30 includes a smart phone, a portable terminal, a mobile terminal, a foldable terminal, a personal digital assistant (PDA), and a PMP. (Portable Multimedia Player) Terminal, Telematics Terminal, Navigation Terminal, Personal Computer, Notebook Computer, Slate PC, Tablet PC, Ultrabook, Wearable Device (Wearable Device, for example, Smartwatch, Smart Glass, HMD (Head Mounted Display), etc.), Wibro terminal, IPTV (Internet Protocol Television) terminal, Smart TV , Digital broadcasting terminals, AVN (Audio Video Navigation) terminals, A/V (Audio/Video) systems, flexible terminals, digital signage devices, and the like.

The object recognition unit 130 recognizes object information included in the screen search request received by the search request receiving unit 140.

In one aspect, the object recognition unit 130 analyzes one of face recognition, gender, age, clothes color, beard, glasses, mask, license plate, vehicle manufacturer, vehicle color, vehicle shape, motorcycle occupant, and bicycle occupant characteristics. Object information can be selectively identified based on metadata. In the object recognition unit 130, a camera 20 or a user terminal 30 including an object recognition function may replace the function.

Metadata is not limited thereto, and is interpreted to include all types of information other than the characteristics or propensity of the object.

The image data extracting unit 150 extracts image data including object information recognized by the object recognition unit from image data captured by a plurality of cameras 20.

In one aspect, the image data extractor 150 receives a running time set value from a search requester, and receives the image data including the extracted object information in units of a running time set value based on the received running time set value. Extract image data.

The running time setting value may be referred to as a setting value for determining how long image data is to be extracted from the images including the object of interest that the search requestor wants to search. For example, the running time setting value can be set in seconds, minutes, hours, and days.

For example, when an object is found for several tens of minutes or more in one channel, it is possible to set the image of the channel to be displayed on the screen only for a time according to the running time setting value set by the search requester.

The screen display unit 160 arranges the image data extracted by the image data extractor 150 in the order of photographing time and displays at least one divided screen.

The screen display unit 160 uses a user interface and/or a graphic user interface stored in the storage unit 180 under the control of a control unit that controls the overall operation of the multi-screen search technology support system 10 to provide various types of menu screens, etc. Content can be displayed.

Here, the content displayed on the screen display unit 160 includes various text or image data (including various information data) and a menu screen including data such as icons, list menus, and combo boxes. Also, the screen display unit 160 may be a touch screen.

In addition, the screen display unit 160 includes a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display. It may include at least one of (Flexible Display), 3D display, e-ink display, and LED (Light Emitting Diode).

In an embodiment, the screen display unit 160 may arrange the image data extracted by the image data extracting unit 150 on a desired channel among multiple search screens. In this case, the search requester may arbitrarily arrange search images or automatically sort them based on the recorded time. The screen display unit 160 reproduces the multiple search result screen.

The screen display unit 160 may further provide an editing function. The screen display unit 160 performs channel readjustment editing by adding, deleting, or relocating channel image data displayed on at least one split screen.

In one aspect, the screen display unit 160 selects the channel-of-interest image data by drag-and-drop method while displaying at least one or more channel image data as a split screen, and selects the selected channel-of-interest image data. You can perform screen editing.

In a state in which a multi-search screen of 3 or more channels is being played on the screen display unit 160, an image channel of interest is selected, a new channel image is added, the order is rearranged, or unnecessary image channels are displayed on the screen display unit 160. It can be excluded from the search screen.

At this time, an image of a channel of interest may be selected through a signal input device such as a mouse, an electronic pen, a touch screen, a motion (movement) sensor input, voice recognition, and various sensors.

In another aspect of the present invention, the screen display unit 160 outputs at least one channel image data list, selects at least one channel of interest image data from the at least one channel image data list, and receives the selected channel image data of interest. You can also edit the screen.

The screen display unit 160 may select a channel of interest by displaying a thumbnail list, which is a part of the channel screen, on the channel image data list while displaying at least one or more channel image data on a split screen.

In an embodiment, the entire channel image data list window may be fixedly displayed at a predetermined position on the multiple search screen. In addition, the screen display unit 160 may drag and drop the selected image data of the channel of interest and arrange it at a location desired by the user.

The screen display unit 160 may also automatically designate the screen arrangement order on the split screen according to the order selected as the channel of interest. In addition, when selecting a channel of interest, screen arrangement information may be manually input.

The screen display unit 160 rearranges the multiple search screens according to the input screen edit contents. Channel images that are not selected as the channel of interest are excluded from the multiple search screen, and the number of split screens for the multiple search is adjusted according to the number of selected channel image data of the selected channel of interest and displayed on the screen.

2 is a block diagram showing a configuration of an apparatus for providing a multi-screen search service having an object recognition function according to another embodiment of the present invention.

As shown in FIG. 2, in the multi-screen search technology support system 10 according to another embodiment of the present invention, an object recognition unit ( 130-1, 130-2..), image data captured from the object recognition cameras 20-1, 20-2...) and metadata generated by the object recognition unit are received.

The multi-screen search technology support system 10 according to an embodiment generates metadata for object information included in the image data received from the system itself by receiving photographed image data from a general camera, or the object recognition camera 20- 1, 20-2...) may also receive captured image data and metadata generated according to object information included in the captured image data.

3 is an exemplary view of a screen in which a search result is displayed on a screen display unit of a multi-screen search technology support system according to an embodiment of the present invention.

As shown in FIG. 3, the screen display unit 160 according to an exemplary embodiment further displays photographing time information of the corresponding image data on image data displayed for each of at least one divided screen.

In addition, the screen display unit 160 further displays camera information that has captured the image data on the image data displayed for each of at least one or more split screens. Accordingly, it is possible to grasp at a glance whether an object to be searched for is photographed by a camera of which channel at any moment, and thus it is possible to accurately search object information without performing a separate search or manipulation.

In an additional aspect of the present invention, the backup unit 170 backs up a search result displayed on the screen display unit 160, that is, image data including divided images of different photographing times of an object to be searched. Conventionally, in order to grasp the movement of a specific object, all captured images of a channel including an object image in a desired time period must be backed up, or all captured images of all channels must be backed up at a specific time period. For example, in the case of 16 channels, 15 unnecessary videos had to be included in the backup file.

However, since the backup unit 170 according to an embodiment extracts only image data including the object to be searched in a specific time period and includes it as a split screen in one screen, a screen including only the image including the object to be searched is included. Data can be created and backed up. Accordingly, the backup time for data backup can be shortened, and the backup capacity can be minimized, resulting in an effect of improving data efficiency.

(Example 1)

The multi-screen search technology support system 10 having an object recognition function according to an embodiment performs an object recognition and metadata generation function. Accordingly, according to an object search request, videos including a corresponding object having a different recording time may be sequentially arranged on one screen and provided to the screen.

Accordingly, the multi-screen search technology support system 10 having an object recognition function according to an exemplary embodiment extracts metadata about objects included in image data received from the camera 20 and stores it together with the image data.

In this case, a user who uses the multi-screen search service according to an embodiment of the present invention may register a channel of interest on the multi-search screen.

Metadata analyzes object characteristics such as face recognition, gender, age, clothing color, beard, glasses, mask, vehicle license plate, vehicle manufacturer, vehicle color, vehicle shape, motorcycle occupant, and bicycle occupant characteristics. It is interpreted to be inclusive.

The search requester can select a channel of interest from a mouse, electronic pen, touch screen, motion (movement) sensor input, voice recognition, and various sensor inputs from the simultaneous search screen window of two or more channels of multiple search screens. In addition, the search requester selects a specific person or object as an object of interest from the image data of the selected channel of interest.

Then, the multi-screen search technology support system 10 according to an embodiment checks the metadata of the object of interest selected by the search requestor. In addition, metadata useful for search among the metadata of the object of interest can be selected as a search criterion.

Thereafter, the search requestor may request a search for stored image data by designating a search date range. In addition, images for which the search is completed may be arranged in a desired channel among multiple search screens. In this case, the search requester may arbitrarily arrange search images or automatically sort them based on the recorded time. In addition, the multi-screen search technology support system 10 reproduces the multi-search screen through the screen display unit.

In a state in which the multiple search screen is displayed, when a search requestor designates a desired channel to perform backup or a backup request is input to backup all channels, the backup unit 170 backs up necessary image data under the control of the search requestor.

(Example 2)

In another embodiment, the multi-screen search technology support system 10 having an object recognition function registers a channel of interest using a thumbnail file and displays videos including a corresponding object having a different recording time according to an object search request on one screen. It can be listed in sequence and provided on the screen.

The multi-screen search technology support system 10 having an object recognition function according to an exemplary embodiment extracts metadata about objects included in image data received from the camera 20 and stores it together with the image data.

In this case, a user who uses the multi-screen search service according to an embodiment of the present invention may register a channel of interest on the multi-search screen.

Metadata analyzes object characteristics such as face recognition, gender, age, clothes color, beard, glasses, mask, license plate, vehicle manufacturer, vehicle color, vehicle shape, motorcycle occupant, and bicycle occupant characteristics. It is interpreted to be inclusive.

The search requestor may search for a thumbnail image by designating an object of interest among recorded images in which metadata about objects is also stored through object recognition. As a search condition, a metadata value can be entered, or an image file containing an object of interest can be used. In addition, it is possible to select the stored image data by specifying the search date and time range.

Then, the searched thumbnail images are placed in a desired channel among the multiple search screens. In this case, the search requestor may arbitrarily arrange the thumbnail images or automatically arrange them based on the recorded time. In addition, the multi-screen search technology support system 10 reproduces the multi-search screen through the screen display unit.

In a state in which the multiple search screen is displayed, when a search requestor designates a desired channel to perform backup or a backup request is input to backup all channels, the backup unit 170 backs up necessary image data under the control of the search requestor.

4 is a flowchart illustrating a method of supporting a multi-screen search technology having an object recognition function according to an embodiment of the present invention.

The multi-screen search technology support method performed in a multi-screen search technology support system including an object recognition unit, a storage unit, an image data extracting unit, and a screen display unit first receives image data captured from a plurality of cameras through a communication unit. (S400), it is determined whether meta data is included in the received image data (S405).

In addition, when the received image data does not contain metadata, that is, when only image data captured by a general camera is received, the object recognition unit recognizes object information included in the image data captured by at least one or more cameras to obtain metadata. Generate (S410).

In one aspect, the multi-screen search technology support system includes image data captured from at least one object recognition camera including an object recognition unit that generates metadata by identifying object information included in the captured image data and the object recognition unit. You can also receive the generated metadata.

That is, when image data and metadata are received together from the object recognition camera (S405), the received image data and metadata are matched together and stored (S420).

At this time, the multi-screen search technology support system according to an embodiment is one of face recognition, gender, age, clothes color, beard, glasses, mask, vehicle license plate, vehicle manufacturer, vehicle color, vehicle shape, motorcycle occupant, and bicycle occupant characteristics. Based on the analyzed metadata, object information is identified, the identified object information is matched with image data, and stored together in a storage unit. Metadata is not limited thereto, and is interpreted to include all types of information other than the characteristics or propensity of the object.

Thereafter, when the screen search request is received (S430), the image data extracting unit identifies object information included in the received screen search request (S440), Image data including identified object information is extracted from among image data stored in a storage unit that matches and stores the data together and provides (S450).

In one aspect, in the step of extracting the image data, a running time set value is received from a search requester (S460), and a running time set value is received based on the running time set value received by receiving the image data including the extracted object information. Image data is extracted in units (S470).

The running time setting value may be referred to as a setting value for determining how long image data is to be extracted from the images including the object of interest that the search requestor wants to search. For example, the running time setting can be set in seconds, minutes, hours, or days.

In addition, the screen display unit arranges the image data extracted by the image data extraction unit in the order of photographing time, and displays and provides at least one divided screen (S480).

In this case, in the step of displaying at least one divided screen, the photographing time information of the corresponding image data is further displayed on the image data displayed for each of the at least one or more divided screens.

In addition, in the displaying of the at least one split screen, camera information on which the corresponding image data is captured is further displayed on the image data displayed for each of the at least one split screen.

Accordingly, it is possible to grasp at a glance whether an object to be searched for is photographed by a camera of which channel at any moment, and thus it is possible to accurately search object information without performing a separate search or manipulation.

According to an aspect of the present invention, after the displaying of the multi-screen search technology, the screen display unit further includes providing an editing function for at least one channel screen displayed as a split screen.

In the providing of the editing function, channel readjustment editing is performed by adding, deleting, or changing the position of channel image data displayed on at least one split screen (S490).

In one aspect, the providing of the editing function includes selecting the channel-of-interest image data in a drag-and-drop method while displaying at least one or more channel image data in a split screen, and the selected channel-of-interest image data. You can edit the screen.

When a multi-search screen of 3 or more channels is being played on the screen display, a video channel of interest can be selected, a new channel video can be added, the order can be rearranged, or unnecessary video channels can be excluded from the multi-search screen displayed on the screen display have.

In addition, the providing of the editing function may include outputting at least one channel image data list, selecting at least one channel image data of interest from the at least one channel image data list, and performing screen editing of the selected channel image data of interest. have.

In an embodiment, the entire channel image data list window may be fixedly displayed at a predetermined position on the multiple search screen. In addition, the screen display unit may place the selected image data of the channel of interest in a desired position by a user by dragging and dropping it.

The step of providing the editing function may also automatically designate the screen arrangement order on the split screen according to the order selected as the channel of interest. In addition, when selecting a channel of interest, screen arrangement information may be manually input.

The screen display unit rearranges the multiple search screens according to the input screen edits. Channel images that are not selected as the channel of interest are excluded from the multiple search screen, and the number of split screens for the multiple search is adjusted according to the number of selected channel image data of the selected channel of interest and displayed on the screen.

According to an additional aspect of the present invention, it is possible to selectively back up the entire image data displayed on a split screen or the image data of a channel of interest selected by the search requester (S500).

Since the backup unit according to an embodiment extracts only image data including the object to be searched in a specific time period and includes it as a split screen on one screen, it creates and backs up screen data including only the image including the object to be searched. can do. Accordingly, the backup time for data backup can be shortened, and the backup capacity can be minimized, resulting in an effect of improving data efficiency.

The above-described method may be implemented as an application or in the form of program instructions that may be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, and the like alone or in combination.

The program instructions recorded in the computer-readable recording medium may be specially designed and constructed for the present invention, and may be known and usable to those skilled in the computer software field.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magnetic-optical media such as floptical disks. media), and a hardware device specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like.

Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the processing according to the present invention, and vice versa.

Although the above has been described with reference to embodiments, those skilled in the art will understand that various modifications and changes can be made to the present invention without departing from the spirit and scope of the present invention described in the following claims. I will be able to.

10: multi-screen search technology support system 20: camera
30: user terminal 110: communication unit
120: image data receiving unit 130: object recognition unit
140: search request receiving unit 150: image data extracting unit
160: screen display unit 170: backup unit
180: storage unit

Claims (15)

An object recognition unit that receives image data captured from at least one camera, recognizes object information included in the received image data, and generates metadata;
A storage unit for matching and storing the received image data and metadata generated by the object recognition unit;
Image data extraction for receiving a screen search request including object information, identifying object information included in the received screen search request, and extracting image data including the identified object information among image data stored in the storage unit part; And
A screen that arranges the image data extracted from the image data extraction unit in the order of shooting time, displays them as at least one split screen, and provides an object recognition-based channel readjustment editing function for at least one or more channel screens displayed as a split screen Including;
The screen display unit,
Provides an editing function for adding new channel image data to at least one split screen of a multi-screen including at least one or more channel screens displayed as a split screen,
A backup unit for backing up image data including divided images of different shooting times of the object to be searched displayed on the screen display unit; further comprising,
The image data extracting unit receives a running time set value from a search requester, and extracts the image data in units of a running time set value based on a running time set value received from the inputted image data including the extracted object information,
The running time setting value is a setting value for determining how long video data is to be extracted from the video including the object of interest that the search requestor wants to search for, a multi-screen search technology support system.
The method of claim 1,
The screen display unit,
A multi-screen search technology support system that provides an editing function for deleting channel image data played on at least one split screen of a multi-screen including at least one or more channel screens displayed as a split screen.
delete The method of claim 1,
The screen display unit,
Multi-screen search, which provides an editing function to place the channel image data in a desired location by dragging and dropping the channel image data played on at least one split screen of a multi-screen including at least one channel screen displayed as a split screen Technical support system.
The method of claim 1,
The screen display unit,
A multi-screen search technology support system for displaying image data captured at different times extracted by the image data extracting unit in the order of photographing time in at least one divided screen.
An image data receiving unit receiving image data captured from at least one object recognition camera including an object recognition unit generating metadata by recognizing object information included in the captured image data and metadata generated by the object recognition unit;
A storage unit for matching and storing image data and metadata received by the image data receiving unit;
Image data extraction for receiving a screen search request including object information, identifying object information included in the received screen search request, and extracting image data including the identified object information among image data stored in the storage unit part; And
A screen that arranges the image data extracted from the image data extraction unit in the order of shooting time, displays them as at least one split screen, and provides an object recognition-based channel readjustment editing function for at least one or more channel screens displayed as a split screen Including;
The screen display unit,
Provides an editing function for adding new channel image data to at least one split screen of a multi-screen including at least one or more channel screens displayed as a split screen,
A backup unit for backing up image data including divided images of different shooting times of the object to be searched displayed on the screen display unit; further comprising,
The image data extracting unit receives a running time set value from a search requester, and extracts the image data in units of a running time set value based on a running time set value received from the inputted image data including the extracted object information,
The running time setting value is a multi-screen search technology support system, characterized in that the search requestor is a setting value for determining how long image data is to be extracted among images including an object of interest desired to be searched.
The method of claim 6,
The screen display unit,
A multi-screen search technology support system that provides an editing function for deleting channel image data played on at least one split screen of a multi-screen including at least one or more channel screens displayed as a split screen.
delete The method of claim 6,
The screen display unit,
Multi-screen search, which provides an editing function to place the channel image data in a desired location by dragging and dropping the channel image data played on at least one split screen of a multi-screen including at least one channel screen displayed as a split screen Technical support system.
The method of claim 6,
The screen display unit,
A multi-screen search technology support system for displaying image data captured at different times extracted by the image data extracting unit in the order of photographing time in at least one divided screen.
In the multi-screen search technology support method performed in a multi-screen search technology support system including an object recognition unit, a storage unit, an image data extracting unit, and a screen display unit,
Generating metadata by recognizing object information included in image data captured by at least one or more cameras by an object recognition unit;
When an image data extraction unit receives a screen search request including object information, the object information included in the received screen search request is identified and the image data photographed by the at least one camera and the metadata generated by the object recognition unit are collected. Extracting image data including the identified object information from among image data stored in a storage unit matching and storing together;
Arranging the image data extracted by the image data extracting unit in the order of photographing time by a screen display unit and displaying at least one divided screen; And
Providing an editing function for at least one channel screen displayed as a split screen by the screen display unit; Including,
Providing the editing function,
Provides an editing function for adding new channel image data to at least one split screen of a multi-screen including at least one or more channel screens displayed as a split screen,
The backup unit backing up image data including divided images of different shooting times of the object to be searched displayed on the screen display unit; further comprising,
The image data extracting unit receives a running time set value from a search requester, and extracts the image data in units of a running time set value based on a running time set value received from the inputted image data including the extracted object information,
The running time setting value is a setting value for determining how long video data is to be extracted from the video including the object of interest that the search requestor wants to search.
The method of claim 11,
Providing the editing function,
A method of supporting a multi-screen search technology, providing an editing function for deleting channel image data played on at least one split screen of a multi-screen including at least one channel screen displayed as a split screen.
delete The method of claim 11,
Providing the editing function,
Providing an editing function in which channel image data played on at least one split screen of a multi-screen including at least one channel screen displayed as a split screen is placed at a desired location by dragging and dropping,
How to support multi-screen search technology.
The method of claim 11,
The displaying of the split screen includes:
Arranging the image data captured at different times extracted by the image data extracting unit in the order of shooting time and displaying them as at least one split screen,
How to support multi-screen search technology.

Images