KR102072022B1 - Apparatus for Providing Video Synopsis Computer-Readable Recording Medium with Program therefore - Google Patents

Abstract

This embodiment discloses a region of interest based video summary device and a computer readable recording medium therefor. In more detail, the image summary apparatus obtains the captured image and generates a determination result of determining whether the object entering the preset ROI within the image is separated. Then, when the image summary apparatus detects an object in the ROI based on the determination result, the image summarization based on the ROI is performed by performing an image summarization to move objects existing in different time zones in the ROI to be located at the same time zone. It provides a video summary device that supports and a computer-readable recording medium therefor.

Description

Apparatus for Providing Video Synopsis Computer-Readable Recording Medium with Program therefore}

The present embodiment relates to a region of interest based video summary device and a computer readable recording medium therefor.

It should be noted that the contents described below merely provide background information related to the present embodiment and do not constitute a prior art.

Recently, due to the rapid development of electronic and communication technologies, the development of camera technology and the spread of smart phones, the amount of video data collected is rapidly increasing, and the amount of information generated from multimedia data such as images, video and audio data is increasing rapidly. It is also exploding.

As the image data increases, technologies for storing the collected images and effectively searching for and reproducing the required image data have been introduced. On the other hand, among these technologies, video summary technology is a technology that allows a person to appear in different time zones in the same space to be reproduced at the same time zone to summarize the recorded video over a long time in a short time.

However, in order to solve the complexity and the time required for the image summary technology, a technique for selectively performing the image summary only for a region of interest in the collected image is required.

In the present embodiment, the image summary apparatus obtains the captured image and generates a determination result of determining whether the object entering the preset ROI within the image is separated. Then, when the image summary apparatus detects an object in the ROI based on the determination result, the complexity of the image summary technology is performed by performing an image summarization to move objects existing in different time zones in the ROI to be located at the same time zone. The main purpose is to eliminate the time required.

In addition, according to another aspect of the present embodiment, the image summary apparatus obtains the captured image and generates a determination result of determining whether the object entering the preset ROI within the image is separated. After that, the image summary apparatus extracts a database of the dwell time information of the object in the region of interest based on the determination result, and solves the complexity and the time required of the image summary technology by performing the image summary using the database dwell time information. The main purpose is to.

The present embodiment includes an image acquisition unit for acquiring a captured image; An image analyzer configured to generate a determination result of determining whether an object that has entered a region of interest (ROI) within the image is separated; And extracting an object dwell image in which the object exists in the region of interest from the image based on the determination result, and using the object dwell image, objects that existed in different time zones in the region of interest are located together at the same time zone. It provides a video summary device comprising an image summary unit for generating a summary image by performing a video summary to shift (Shift).

In addition, according to another aspect of the present embodiment, the image acquisition unit for obtaining a photographed image; An image storage unit storing the image and related information related to the image; A determination result of determining whether the object has entered the region of interest set in the image is generated, and a time at which the object enters the region of interest, a time at which the object has left, and a stay of the object based on the determination result An image analyzer extracting dwell time information including a timeline; A controller for controlling the dwell time information to be databased in the image storage unit; And extracting an object dwell image corresponding to the dwell time from the image based on the dwell time information databased in the image storage unit when receiving a command to perform a summary of the image, and using the object dwell image. It provides an image summary device comprising an image summary unit for generating a summary image performing the image summary.

In addition, according to another aspect of the present embodiment, the data processing device, the image acquisition process of acquiring the captured image; A determination result generation step of generating a determination result of determining whether the object entering the preset ROI in the image is separated; An extraction process of extracting an object residence image in which the object exists in the ROI of the image based on the determination result; And a program for realizing an image summarization process of generating a summary image by performing image summarization to move objects existing in different time zones in the ROI to be located at the same time zone using the object dwell image. A readable recording medium is provided.

In addition, according to another aspect of the present embodiment, the data processing device, the image acquisition process of acquiring the captured image; A determination result generation step of generating a determination result of determining whether the object entering the preset ROI in the image is separated; An extraction process of extracting residence time information including a time at which the object enters the ROI, a time at which the object leaves, and a dwell time of the object based on the determination result; A control process of controlling the dwell time information to be a database; An image extraction process of extracting an object dwell image corresponding to the dwell time from the dwell based on the dwell time information when receiving a command to perform an image summary; And a program for realizing an image summarization process of generating a summary image by performing image summarization to move objects existing in different time zones in the ROI to be located at the same time zone using the object dwell image. A readable recording medium is provided.

In the present embodiment, the image summary apparatus obtains the captured image and generates a determination result of determining whether the object entering the preset ROI within the image is separated. Then, when the image summary apparatus detects an object in the ROI based on the determination result, the complexity of the image summary technology is performed by performing an image summarization to move objects existing in different time zones in the ROI to be located at the same time zone. There is an effect that can eliminate the time required.

In addition, according to another aspect of the present embodiment, the image summary apparatus obtains the captured image and generates a determination result of determining whether the object entering the preset ROI within the image is separated. After that, the image summary apparatus extracts a database of the dwell time information of the object in the region of interest based on the determination result, and solves the complexity and the time required of the image summary technology by performing the image summary using the databased dwell time information. The main purpose is to.

1 is a block diagram schematically illustrating an image summary device according to a first embodiment of the present invention.
2 is a block diagram schematically illustrating an image summary device according to a second embodiment of the present invention.
3 is a flowchart illustrating a method of performing image summarization by an image summary apparatus according to a first embodiment of the present invention.
FIG. 4 is a flowchart illustrating a method of performing an image summary by using the database dwell time information by the image summary device according to the second embodiment of the present invention.
FIG. 5 is a flowchart for describing a method of performing an image summary based on importance information for each region of interest by an image summary apparatus according to a second exemplary embodiment of the present invention.
FIG. 6 is an exemplary view illustrating a summary image generated by an image summary apparatus according to an embodiment of the present invention.

Hereinafter, the present embodiment will be described in detail with reference to the accompanying drawings.

1 is a block diagram schematically illustrating an image summary device 100 according to a first embodiment of the present invention.

As illustrated in FIG. 1, the image summary apparatus 100 according to the first embodiment of the present invention may include an image acquisition unit 110, an image storage unit 120, an ROI setting unit 130, and an importance setting unit 140. ), An image analyzer 150, an image summary unit 160, and an image reproducing unit 170. In the image summary device 100 according to the first embodiment of the present invention, the image summary device 100 may include an image acquisition unit 110, an image storage unit 120, an ROI setting unit 130, and an importance setting unit 140. ), But it is described as including only the image analysis unit 150, the image summary unit 160 and the image playback unit 170, but is not necessarily limited thereto.

The image obtaining unit 110 obtains an image captured by the image capturing apparatus from an image capturing apparatus (not shown) interlocked with the image summary apparatus 100. That is, the image acquisition unit 110 acquires an image of a predetermined region photographed from an image photographing apparatus installed at a specific point. Here, the captured image may be one frame or may include an image including a plurality of frames. On the other hand, the image acquisition unit 110 may receive an image captured by the image capturing apparatus in real time, but may also obtain an image that is previously photographed and stored in the image capturing apparatus at predetermined intervals. In addition, in the present embodiment, it is specified that the image acquisition unit 110 acquires an image captured by the image capturing apparatus, but is not limited thereto. The image summary apparatus 100 includes a module for capturing an image. The module may be used to directly acquire the captured image.

The image storage unit 120 stores the image acquired by the image acquisition unit 110 and provides the stored image in the process of performing the image summary. That is, the image storage unit 120 stores the captured image and stores the captured image when requesting the captured image from a device such as the ROI setting unit 130, the image analyzer 150, and the image summary unit 160. Provide video. On the other hand, the image storage unit 120 may classify and store the obtained image according to criteria such as by image capturing device, period, time zone, and the like, and pre-stores image related information added in the process of performing image summary. The image may be matched and stored. Meanwhile, the image received by the image storage unit 120 may further include photographing time information at which the image is captured.

The ROI setting unit 130 receives a selection signal for selecting a specific region in the image and sets the specific region as the ROI based on the received selection signal. That is, the ROI setting unit 130 receives a selection signal for selecting a specific area in the image from the user using the user interface unit (not shown) and sets the specific area as the ROI. Thereafter, the ROI setting unit 130 stores and manages database settings of the ROI.

In addition, when the ROI in the specific image is set, the ROI setting unit 130 sets the same region as the ROI in another image corresponding to the specific image, for example, another image received from the image photographing apparatus in which the specific image is captured. Control as possible. Meanwhile, the ROI setting unit 130 may operate the image storage unit 120 to provide an image in which the ROI is set by providing a setting value for the ROI to the image storage unit 120. The 130 may generate an image in which the ROI is directly set based on the image received from the image storage unit 120 and provide the image to the image analyzer 150.

When the plurality of ROIs are set in the image by using the ROI setting unit 130, the importance setting unit 140 sets the importance of the plurality of ROIs. That is, when the plurality of ROIs are set, the importance setting unit 140 receives a selection signal that selects the importance of the plurality of ROIs from the user, and selects the importance of the plurality of ROIs based on the received selection signals. Set it. Thereafter, the importance setting unit 140 provides the image storage unit 120 with importance setting values for the plurality of ROIs, and the corresponding information in the process of performing the image summary based on the ROI by the image storage unit 120. To provide.

The image analyzer 150 generates a determination result of determining whether an object that has entered a region of interest (ROI) set in the image has left. That is, when the image analyzer 150 receives a request signal for performing image summarization, the image analyzer 150 receives an image of which an ROI is set from the image storage unit 120. Thereafter, the image analyzer 150 determines whether the object entering the region of interest set in the image has left. Meanwhile, in the present embodiment, when the image analysis unit 150 receives a request signal for performing the image summarization operation, it is specified that the process of generating the determination result is performed. Immediately afterwards, the process of generating the judgment result may be repeatedly performed.

Meanwhile, when the image analyzer 150 receives an image in which the ROI is set from the image storage unit 120, the image analyzer 150 searches for an object existing in the ROI. That is, the image analyzer 150 extracts the shape of each object from the ROIs in the continuous image frames constituting the received image. In this case, the method of extracting the shape of the object by the image analyzer 150 is not limited to a specific method, and various algorithm techniques may be used. For example, the image analyzer 150 may check a pixel value of a pixel in the ROI to remove a background pixel of a background having no specified movement, and extract the shape of an object through this. Also, the image analyzer 150 compares the received image with the unique background information based on the unique background information for each image set in the image analyzer 150, and based on the comparison result, the image analyzer 150 exists in the image. You can also extract the shape of an object. Meanwhile, in the process of extracting an object, the image analyzer 150 may further extract coordinate value information in an image in which the object is located.

The image analyzer 150 extracts the shape of the object existing in the ROI and determines whether the object in the ROI enters or leaves based on the extracted information. That is, the image analyzer 150 determines whether the object in the ROI is detected for each consecutive image frame, and if the object is detected in the ROI, the image analyzer 150 determines that the object enters the ROI. In addition, when the object is not detected in the ROI, the image analyzer 150 determines that the object has left the ROI. In this way, the image analyzer 150 generates a determination result of determining whether the object entering the region of interest set in the image has left.

On the other hand, if it is determined that the object enters the ROI through the process of extracting the shape of the object, the image analyzer 150 may enter the ROI, the departure time of the departure from the ROI, and the object of interest. The movement of the object from the time of entering the area to the time of leaving the area of interest is further determined. That is, the image analyzer 150 determines the entry and exit time of the object in the ROI by using time information included in the captured image, and additionally extracts coordinate value information in the process of extracting the object in the ROI. Further determine the movement of the object using. Meanwhile, the method of tracking the movement of the object in the ROI is not limited to a specific method. Thereafter, the image analyzer 150 includes the additionally determined information in the determination result and transmits the information to the image summary unit 160.

The image summary unit 160 extracts an object stay image in which an object exists in the ROI from among the images acquired by the image acquisition unit 110 based on the determination result received from the image analyzer 150. That is, the image summary unit 160 extracts an image frame in which an object exists in a region of interest among successive image frames based on the determination result received from the image analyzer 150. For example, when it is determined that an object enters a region of interest in a specific image based on the determination result received from the image analyzer 150, the image summary unit 160 extracts the image as an object stay image. In this case, the object dwell image includes time information on which the corresponding object dwell image is further extracted based on the determination result generated by the image analyzer 150.

The image summary unit 160 performs image summarization using the object stay image, and generates a summary image. On the other hand, video synopsis shifts people who appear in different time zones in the same space so that they can be played together at the same time zone, and displays the time when the corresponding people entered. Here is a summary. That is, the image summary unit 160 generates a summary image by performing an image summary for moving objects existing in different time zones in the ROI to be located at the same time zone using the object stay image.

Meanwhile, when the object is detected in the ROI in the image acquired by the image acquisition unit 110, the image summary unit 160 enters the ROI based on the determination result received from the image analyzer 150. If the object is not detected in the ROI, it is determined that the object has left the ROI and the image summary is stopped.

Hereinafter, an operation of performing image summarization by using the object stay image by the image summarizing unit 160 will be described. The image summary unit 160 extracts an object existing in the ROI in the object staying image, and in the process of extracting the object, coordinate value information in the image where the object is located and a time when the object exists in the ROI, Time information of the object retention image including the object is additionally extracted. Thereafter, the image summary unit 160 generates metadata that divides the extracted object according to the object type and the photographed time information.

In addition, the image summarizing unit 160 corresponds to the object staying image among the basic image set by the image capturing unit 160, that is, the image including only the background information continuously positioned in the image excluding the object. The image is extracted, and the image summary of moving the extracted object to the basic image is performed. For example, the image summarizing unit 160 sets the object such that the objects extracted from the images that the object first enters the region of interest based on the metadata are positioned at the point where the object is located among the coordinate values of the base image. Move it. Similarly, the image summarizing unit 160 moves the objects extracted from the following images of the image in which the object first enters the ROI, based on the time information included in the metadata, to separate base images for each elapsed time. Perform iteratively. Then, the image summary unit 160 controls the plurality of basic images generated through the image summarization process to be connected and reproduced, so that the objects existing in different time zones in the ROI are located together in the same time zone and the same space. And generate a summary image to be played back. Meanwhile, in the method of generating the summary image by performing the image summarization by the image summarizing unit 160, various methods may be used.

In addition, the image summary unit 160 controls to further display at least one of an entry time, a departure time, and a remaining time for each object located in the summary image. That is, when the summary image is reproduced based on the determination result received from the image analyzer 150, only the entry time or the departure time for each object located in the summary image is continuously maintained. The display may be controlled to be displayed, and the remaining time of the object may be changed and displayed as time passes.

In addition, when the summary image is reproduced, the image summary unit 160 controls to display an outline surrounding the periphery of the preset region of interest so that the region of interest in the image is highlighted and reproduced. In this case, when the plurality of ROIs are set and the importance of the plurality of ROIs is set, the image summary unit 160 may modify a thickness and a color of the outline surrounding the ROI according to the importance of the ROIs. In this case, the ROI may be controlled to clearly distinguish the ROIs of each importance within the image.

On the other hand, when the image summary unit 160 receives selection information on the ROI having a specific importance, the image summary unit 160 may perform image summarization only on the ROI corresponding to the selection information. That is, when the plurality of ROIs are set and the importance of the plurality of ROIs is set, the image summary unit 160 receives selection information about the ROI having a specific importance from the user and based on the interests The image summarization may be performed using the object stay image corresponding to the region.

On the other hand, the image summary unit 160 according to the present embodiment preferably generates a summary image at a predetermined period, but is not necessarily limited thereto. For example, the image summary unit 160 may perform image summarization when receiving a request signal for generating a summary image from the user.

The image reproducing unit 170 interworks with the image summarizing unit 160 to reproduce the summary image. That is, the image reproducing unit 170 receives the summary image from the image summary unit 160 and reproduces the summary image according to a preset standard.

2 is a block diagram schematically illustrating an image summary device 200 according to a second embodiment of the present invention.

As illustrated in FIG. 2, the image summary apparatus 200 according to the second embodiment of the present invention may include an image acquisition unit 110, an image storage unit 120, an ROI setting unit 130, and an importance setting unit 140. ), An image analyzer 150, an image summary unit 160, an image reproducing unit 170, and a controller 210. Components included in the image summary apparatus 200 according to the second embodiment of the present invention are not necessarily limited thereto. Meanwhile, the image summary apparatus 200 according to the second embodiment of the present invention includes the same components as the image summary apparatus according to the first embodiment illustrated in FIG. 1 except for the controller 210, respectively. There are differences in the functions performed by the components of and the manner in which the video summary is performed. Therefore, in FIG. 2, the difference will be described in detail.

The image acquisition unit 110, the image storage unit 120, the ROI setting unit 130, and the importance setting unit 140 may set the image acquisition unit 110, the image storage unit 120, and the ROI specified in FIG. 1. The same function as the unit 130 and the importance setting unit 140, and detailed description of the corresponding devices will be omitted. However, the image storage unit 120 according to the present exemplary embodiment includes a time in which an object enters a region of interest in an image extracted from the image analyzer 150, a time in which the object leaves, and a timeline of the object. Time information is stored in a database for each region of interest. That is, the image storage unit 120 matches and stores the image corresponding to the dwell time information among previously stored images based on the dwell time information. Thereafter, when the image storage unit 120 receives a command to perform a summary of the image from the user, the image storage unit 120 provides the database summary time information to the image summary unit 160, through which the image summary unit 160 provides the residence time information. Operate to perform an image summary based on.

The image analyzer 150 generates a determination result of determining whether the object entering the region of interest set in the image is separated. That is, when the image analyzer 150 receives a request signal for performing image summarization, the image analyzer 150 receives an image of which an ROI is set from the image storage unit 120. Thereafter, the image analyzer 150 determines whether the object entering the region of interest set in the image has left.

Meanwhile, when the image analyzer 150 receives an image in which the ROI is set from the image storage unit 120, the image analyzer 150 searches for an object existing in the ROI. That is, the image analyzer 150 extracts the shape of each object from the ROIs in the continuous image frames constituting the received image. In this case, the method of extracting the shape of the object by the image analyzer 150 is the same as the method of extracting the shape of the object by the image analyzer 150 illustrated in FIG. 1, and a detailed description thereof will be omitted.

The image analyzer 150 extracts the shape of the object existing in the ROI and determines whether the object in the ROI enters or leaves based on the extracted information. That is, the image analyzer 150 determines whether the object in the ROI is detected for each consecutive image frame, and if the object is detected in the ROI, the image analyzer 150 determines that the object enters the ROI. In addition, when the object is not detected in the ROI, the image analyzer 150 determines that the object has left the ROI. In this way, the image analyzer 150 generates a determination result of determining whether the object entering the region of interest set in the image has left.

On the other hand, if it is determined that the object enters the ROI in the image based on the determination result, the image analyzer 150 according to the present embodiment has time, the departure time and the object stayed in the ROI. Extraction of residence time information including residence period That is, the image analyzer 150 extracts the entry, departure time and dwell time of the object in the ROI by using the time information included in the captured image, and additionally adds an image corresponding to the dwell time based on this. Analyze

When the dwell time information is extracted from the image analyzer 150, the controller 210 controls the extracted dwell time information to be databased in the image storage unit 120. That is, the controller 210 controls to transmit the dwell time information to the image storage unit 120, and through this, the image storage unit 120 stores the image in the image storage unit 120 based on the dwell time information. The image corresponding to the dwell time information is controlled to be databased by matching the dwell time information.

When the image summarizing unit 160 receives the execution command for the image summarization, the image summarizing unit 160 extracts the object dwell image corresponding to the dwell period from among the images already stored based on the dwell time information databased in the image storage unit 120. do. Meanwhile, the execution command received by the image summary unit 160 may be an image summary execution command for an object located in the ROI in a specific time zone or when a plurality of ROIs and importance of each ROI are set. It may also be an image summary execution command for.

The image summary unit 160 performs image summarization using the object stay image, and generates a summary image. That is, the image summary unit 160 generates a summary image by performing an image summary for moving objects existing in different time zones in the ROI to be located at the same time zone using the object stay image.

On the other hand, when the object is detected in the region of interest in the image acquired by the image acquirer 110, the image summary unit 160 detects the object in the region of interest based on the residence time information databased in the image storage unit 120. The image summary is determined by entering, and when the object is not detected in the region of interest, the image summary is stopped by determining that the object has left the region of interest.

Meanwhile, the operation of the image summary unit 160 according to the present embodiment to perform the image summary using the object stay image is identical to the operation of the image summary unit 160 shown in FIG. 1 to perform the image summary. Is omitted.

Meanwhile, when the plurality of ROIs are set and the importance of the plurality of ROIs is set, the image summary unit 160 receives selection information about the ROI having a specific importance from the user and based on the interests The image summarization may be performed using the object stay image corresponding to the region. That is, when the image summarizing unit 160 receives selection information on the ROI having a specific importance from the user, the image summarizing unit 160 may perform the image summarization process by using the dwell time information in the ROI corresponding to the importance.

The image reproducing unit 170 interworks with the image summarizing unit 160 to reproduce the summary image. That is, the image reproducing unit 170 receives the summary image from the image summary unit 160 and reproduces the summary image according to a preset standard.

3 is a flowchart illustrating a method of performing image summarization by the image summary apparatus 100 according to the first embodiment of the present invention.

As shown in FIG. 3, the method of performing the image summarization by the image summary apparatus 100 according to the first embodiment of the present invention begins with the process of acquiring the captured image by the image summary apparatus 100 (S300). ). That is, an image captured by the image capturing apparatus is obtained from an image capturing apparatus linked with the image summary apparatus 100. Here, the captured image may be one frame or may include an image including a plurality of frames.

When the image summary apparatus 100 receives an image summary task command (S302), the image summary apparatus 100 determines whether an object enters a region of interest set in the image (S304). That is, when the image summary apparatus 100 receives an image summary task command to generate a determination result of determining whether the object entering the region of interest has deviated, whether the object enters the region of interest in the image acquired from the image capturing apparatus. Judge. Meanwhile, the image summary apparatus 100 extracts the shape of each object from the ROIs in the continuous image frames constituting the received image, and determines whether the object enters the ROI based on the extracted information. For example, the image summary apparatus 100 may determine whether an object in the ROI is detected for each consecutive image frame, and if the object is detected in the ROI, the image summary apparatus 100 determines that the object enters the ROI.

If it is determined that the object has entered the ROI set in the image, the image summary apparatus 100 tracks the movement of the object in the ROI (S306). That is, if it is determined in step S304 that the object enters the ROI, the image summary apparatus 100 uses the time information included in the captured image to depart from the entry time and the ROI of the ROI. Determine the departure time. In addition, the image summary apparatus 100 may further determine the movement of the object by using the extracted coordinate value information in the process of extracting the object in the ROI.

The image summary apparatus 100 determines whether the object in the region of interest is separated (S308), and when it is determined that the object in the region of interest is separated, the image summary apparatus 100 generates a determination result that determines whether the object has entered the region of interest (S310). ). That is, the image summary apparatus 100 checks whether the object in the ROI is detected for each consecutive image frame by using the process of tracking the movement of the object in the ROI in step S306, and if the object is not detected in the ROI. In this case, it is determined that the object has left the region of interest. In this way, the image summary apparatus 100 generates a determination result of determining whether the object entering the region of interest set in the image has left.

The image summary apparatus 100 determines whether the predetermined cycle has elapsed in the image summary apparatus 100 (S312), and when it is determined that the predetermined cycle has elapsed, the image summary apparatus 100 determines the object staying image based on the determination result generated in step S310. Extract it (S314). That is, the image summary apparatus 100 extracts the object dwell image based on a determination result generated when a predetermined period, for example, a day or a specific time arrives. On the other hand, when it is determined that the object enters the ROI in the specific image, the image summary apparatus 100 extracts the corresponding image as the object staying image.

The image summary apparatus 100 performs an image summary using the object stay image, and generates a summary image through this (S316). That is, the image summary apparatus 100 generates a summary image by performing an image summary for moving objects existing in different time zones in the ROI to be located at the same time zone using the object staying image.

On the other hand, when the object is detected in the region of interest in the image based on the determination result generated through step S310, the image summary apparatus 100 determines that the object has entered the region of interest and performs image summarization. If the object is not detected, the image summary is stopped by determining that the object has left the ROI. Thereafter, the image summary apparatus 100 controls the plurality of basic images generated through the image summarization process to be connected and reproduced as one, whereby objects existing in different time zones in the ROI are located together in the same time zone and the same space. And generate a summary image to be played back.

The image summary apparatus 100 receives the summary image generated in step S316 and reproduces the received image (S318).

In FIG. 3, steps S300 to S318 are described as being sequentially executed, but are not necessarily limited thereto. That is, since the steps described in FIG. 3 may be applied by changing or executing one or more steps in parallel, FIG. 3 is not limited to the time series order.

4 is a flowchart illustrating a method in which the image summary apparatus 200 according to the second embodiment of the present invention performs image summarization using a database of dwell time information.

As shown in FIG. 4, in the method of performing a video summary by using the database dwell time information, the image summary device 200 according to the second embodiment of the present invention first captures an image captured by the image summary device 200. Starting from the process of obtaining the (S400). That is, an image captured by the image capturing apparatus is obtained from an image capturing apparatus linked with the image summary apparatus 200. Here, the captured image may be one frame or may include an image including a plurality of frames.

The image summary apparatus 200 generates a determination result of determining whether the object that has entered the ROI set in the image is separated (S402). On the other hand, when receiving the image summary work command, the image summary apparatus 200 extracts the shape of an object existing in the ROI, and determines whether the object in the ROI enters or leaves based on the extracted information. That is, the image summary apparatus 200 determines whether the object in the ROI is detected for each consecutive image frame, and if the object is detected in the ROI, the image summary apparatus 200 determines that the object enters the ROI. In addition, when the object is not detected in the ROI, the image summary apparatus 200 determines that the object has left the ROI. In this way, the image summary apparatus 200 generates a determination result of determining whether the object entering the region of interest set in the image has left.

The image summary apparatus 200 extracts the dwell time information based on the determination result generated in step S402 (S404). That is, when it is determined that an object enters a region of interest in an image based on the determination result, the image summary apparatus 200 includes a stay including an entry time, an escape time, and a period of stay of an object in the region of interest. Extract time information.

When the dwell time information is extracted through the step S404, the image summary apparatus 200 controls the extracted dwell time information to be a database (S406). That is, the image summary apparatus 200 controls the image corresponding to the dwell time information among the images previously stored in the image summary device 200 to be matched with the dwell time information based on the dwell time information.

When receiving the execution command for the image summary (S408), the image summary apparatus 200 extracts the object residence image based on the databased residence time information (S410). That is, when receiving the execution command for the image summary, the image summary apparatus 200 checks the databased dwell time information and extracts the object dwell image corresponding to the dwell time information. Meanwhile, the execution command received by the image summary apparatus 200 may be an image summary execution command for an object located in the ROI in a specific time zone or when a plurality of ROIs and importance of each ROI are set. It may also be an image summary execution command for.

The image summary apparatus 200 performs an image summary using the object stay image, and generates a summary image through this (S412). That is, the image summary apparatus 200 generates a summary image by performing an image summary for moving objects existing in different time zones in the ROI to be located at the same time zone using the object stay image. On the other hand, when the object is detected in the region of interest in the image based on the databased dwell time information, the image summary apparatus 200 determines that the object has entered the region of interest and performs image summarization. If detected, the image summary is stopped by determining that the object has left the ROI.

In FIG. 4, steps S400 to S412 are described as being sequentially executed, but are not necessarily limited thereto. That is, since the steps described in FIG. 4 may be applied by changing the execution or one or more steps in parallel, FIG. 4 is not limited to the time series order.

FIG. 5 is a flowchart illustrating a method of performing an image summary based on importance information for each region of interest by the image summary apparatus 200 according to the second embodiment of the present invention.

As shown in FIG. 5, in the method of performing an image summary on the basis of the importance information for each region of interest, the image summary apparatus 200 according to the second embodiment of the present invention first displays an image captured by the image summary apparatus 200. Starting from the process of obtaining (S500). That is, an image captured by the image capturing apparatus is obtained from an image capturing apparatus linked with the image summary apparatus 200. Here, the captured image may be one frame or may include an image including a plurality of frames.

When receiving the selection information selecting the plurality of ROIs from the user, the image summary apparatus 200 sets a plurality of ROIs in the image based on the selection information. Thereafter, when receiving the setting information for setting the importance of the plurality of ROIs, the image summary apparatus 200 sets the importance for each of the plurality of ROIs based on the setting information (S502).

The image summary apparatus 200 generates a determination result of determining whether the object entering the region of interest set in the image is left at step S504. On the other hand, when receiving the image summary work command, the image summary apparatus 200 extracts the shape of an object existing in the ROI, and determines whether the object in the ROI enters or leaves based on the extracted information. That is, the image summary apparatus 200 determines whether the object in the ROI is detected for each consecutive image frame, and if the object is detected in the ROI, the image summary apparatus 200 determines that the object enters the ROI. In addition, when the object is not detected in the ROI, the image summary apparatus 200 determines that the object has left the ROI. In this way, the image summary apparatus 200 generates a determination result of determining whether the object entering the region of interest set in the image has left.

The image summary apparatus 200 extracts the dwell time information based on the determination result generated in step S504, and makes a database of the extracted dwell time information (S506). That is, when it is determined that the object enters the ROI in the image based on the determination result, the image summary apparatus 200 includes a stay including an entry time, an escape time, and a stay period of the object in the ROI. Extract time information. Thereafter, the image summary device 200 controls the image corresponding to the dwell time information among the images previously stored in the image summary device 200 to be databased by matching the dwell time information based on the dwell time information.

When receiving the image summary execution command for the ROI having a specific importance (S508), the image summary apparatus 200 extracts the object dwell image based on the dwell time information of the ROI corresponding to the specific importance (S510). . That is, when the image summary apparatus 200 receives an image summary execution command for a region of interest having a specific importance, the image summary apparatus 200 determines the residence time information corresponding to the region of interest from the databaseed residence time information, and determines the determined residence time. Based on the information, the object retention image is extracted. Meanwhile, the image summary execution command received by the image summary apparatus 200 may further include selection information about an image received from a specific time zone and a specific image photographing device, as well as selection information about a region of interest having a specific importance.

The image summary apparatus 200 performs an image summary using the object stay image, and generates a summary image through this (S512). That is, the image summary apparatus 200 generates a summary image by performing an image summary for moving objects existing in different time zones in the ROI to be located at the same time zone using the object stay image.

In FIG. 5, steps S500 to S512 are described as being sequentially executed, but are not necessarily limited thereto. That is, since the steps described in FIG. 5 may be applied by changing the execution or one or more steps in parallel, FIG. 5 is not limited to the time series order.

As described above, the method of performing the image summarization described in FIGS. 3, 4 and 5 may be implemented in a program and recorded in a computer-readable recording medium. The computer-readable recording medium having recorded thereon a program for implementing the method of performing the image summarization according to the present embodiment includes all kinds of recording devices storing data that can be read by a computer system. Examples of such computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, and are implemented in the form of a carrier wave (for example, transmission over the Internet). It includes being. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. Also, functional programs, codes, and code segments for implementing the present embodiment may be easily inferred by programmers in the art to which the present embodiment belongs.

6 is an exemplary view illustrating a summary image generated by an image summary apparatus according to an embodiment of the present invention. 6 may be an image summary device 100 according to a first embodiment of the present invention or an image summary device 200 according to a second embodiment of the present invention.

6A is an exemplary diagram illustrating a summary image when one ROI is set in an image. In FIG. 6A, the vicinity of the entrance and exit of the tourist destination is set as the region of interest. In addition, although FIG. 6A illustrates that the vehicle and the object exist outside the ROI, only background information continuously positioned in the image except the object is displayed. In addition, although FIG. 6A illustrates that the summary image is displayed in the form of a still image, the summary image is substantially implemented in the form of a reproduction image in which movement of an object in the ROI is reproduced.

As shown in FIG. 6A, objects that existed in different time zones in the ROI are displayed together in the same time zone through the image summary process. On the other hand, the summary image is displayed with at least one of the entry time, departure time and the remaining time for each object located in the summary image. That is, only the entry time or the departure time for each object may be continuously displayed in the summary image, and the remaining time of the object may be displayed as the time elapses. In addition, in the summary image, an outline surrounding the periphery of the preset ROI is additionally displayed to reproduce the ROI in the image.

6B is an exemplary diagram illustrating a summary image when a plurality of ROIs in an image are set and importance is set for each ROI. On the other hand, in Fig. 6b, the vicinity of the entrance and the crosswalk in front of the tourist destination is set as the area of interest, respectively, and the vicinity of the entrance and exit of the tourist destination is set higher in importance than the front of the crosswalk.

As shown in FIG. 6B, objects existing in different time zones in the ROI are displayed together in the same time zone through the image summary process. In addition, the summary image displays at least one of an entry time, an escape time, and a remaining time for each object located in the summary image. Meanwhile, in FIG. 6B, the object moved through the image summary does not exist in front of the pedestrian crossing set as the region of interest, but this is merely an embodiment for explaining that a plurality of regions of interest may be set. In front of the crosswalk set as the region of interest, there are objects moved through the image summary. In addition, although FIG. 6B illustrates that image summarization is performed on a plurality of ROIs, when a specific ROI is selected from a plurality of ROIs, the image summarization may be performed only on the selected ROI.

Meanwhile, in the summary image, an outline surrounding the periphery of the preset ROI is additionally displayed to reproduce the ROI in the image. In addition, depending on the importance of the preset ROI, the thickness and color of the outline surrounding the ROI are expressed differently so that the ROIs of each importance are clearly displayed in the image. Meanwhile, in FIG. 6B, the thickness of the outline is displayed to be thicker than the region of interest in the area of the front side of the crosswalk in which the region of interest is located near the entrance and exit of the tourist attraction.

The above description is merely illustrative of the technical idea of the present embodiment, and those skilled in the art to which the present embodiment belongs may make various modifications and changes without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of the present embodiment but to explain, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present embodiment.

100: video summary device according to the first embodiment
110: image acquisition unit 120: image storage unit
130: region of interest setting unit 140: importance setting unit
150: image analysis unit 160: image summary unit
170: video playback unit
200: The image summary device according to the second embodiment
210: control unit

Claims (18)

An image acquisition unit for acquiring a captured image;
An image analyzer configured to generate a determination result of determining whether an object that has entered a region of interest (ROI) within the image is separated; And
Based on the result of the determination, the object dwell image in which the object exists in the region of interest is extracted from the image, and the objects existing in different time zones in the region of interest are located together in the same time zone using the object dwell image. Shifting image summary is performed to generate a summary image, and when the summary image is played back, only the entry time or the departure time of each object located in the summary image is continuously displayed or the time elapsed. Image summarization control to display the remaining time of the object is changed according to
Video summary device comprising a. The method of claim 1,
The video summary unit,
When the object is detected in the ROI based on the determination result, the object is determined to enter the ROI, the image summary is performed, and when the object is not detected in the ROI, the object And the image summary device stops the summary of the image by determining that it has left the region of interest. delete The method of claim 1,
The video summary unit,
And, when the summary image is reproduced, controls to display an outline surrounding the periphery of the ROI. The method of claim 1,
When a plurality of regions of interest are set, the apparatus may further include an importance setting unit for setting the importance of the plurality of regions of interest.
The image summarizing unit, when receiving selection information regarding a specific importance level, the image summarizing apparatus, performing the image summarization on a region of interest corresponding to the specific importance level. The method of claim 1,
The video summary unit,
And the summary image is generated at a predetermined cycle. The method of claim 1,
The image analyzer,
If it is determined that the object enters the ROI, the object enters the ROI, the departure time from the ROI, and the object enters the ROI from the ROI. And further determine the movement of the object to the viewpoint. The method of claim 1,
And an image reproducing unit interworking with the image summary unit to reproduce the summary image. An image obtaining unit obtaining a photographed image;
An image storage unit storing the image and related information related to the image;
A determination result of determining whether the object has entered the region of interest set in the image is generated, and a time at which the object enters the region of interest, a time at which the object has left, and a stay of the object based on the determination result An image analyzer extracting dwell time information including a timeline;
A controller for controlling the dwell time information to be databased in the image storage unit; And
When receiving an execution command for the image summary, the object dwell image corresponding to the dwell time is extracted from the image based on the dwell time information databased in the image storage unit, and the image is obtained using the dwell image. A summary image is generated by performing a summary, and when the summary image is played, only the entry time or the departure time for each object located in the summary image is continuously displayed or the remaining time of the object over time is generated. Image control unit to control the display to be changed
Video summary device comprising a. The method of claim 9,
The video summary unit,
When the object is detected in the ROI based on the dwell time information, the object is determined to have entered the ROI, the image summary is performed, and when the object is not detected in the ROI, And the image summary device is stopped by determining that an object has left the ROI. The method of claim 9,
When a plurality of regions of interest are set, the apparatus may further include an importance setting unit for setting the importance of the plurality of regions of interest.
The image summarizing unit, when receiving selection information regarding a specific importance level, performs the image summarization based on the dwell time information of the ROI corresponding to the specific importance level. Data processing devices,
Image acquisition process of acquiring the captured image;
A determination result generation step of generating a determination result of determining whether the object entering the preset ROI in the image is separated;
An extraction process of extracting an object residence image in which the object exists in the ROI of the image based on the determination result; And
A summary image is generated by moving the objects that existed in different time zones in the ROI to be located at the same time zone using the object stay image to generate a summary image. Image summarization process that controls only the entry time or departure time for each object to be displayed continuously or the remaining time of the object changes to be displayed over time
A computer-readable recording medium having recorded thereon a program for realizing the problem. The method of claim 12,
The determination result generation process,
If it is determined that the object has entered the ROI, the object enters the ROI, the departure time has left the ROI, and the object has left the ROI from the entry point of the ROI. A computer-readable recording medium having recorded thereon a program for realizing, further comprising the step of further determining the movement of the object to a viewpoint. The method of claim 12,
The video summary process,
If the object is detected in the ROI based on the determination result, determining that the object enters the ROI and performing the image summary; And
And determining that the object has left the region of interest when the object is not detected in the region of interest, and stopping the summary of the image. The method of claim 12,
And a summary image reproducing process of reproducing the summary image. Data processing devices,
Image acquisition process of acquiring the captured image;
A determination result generation step of generating a determination result of determining whether the object entering the preset ROI in the image is separated;
An extraction process of extracting residence time information including a time at which the object enters the ROI, a time at which the object leaves, and a dwell time of the object based on the determination result;
A control process of controlling the dwell time information to be a database;
An image extraction process of extracting an object dwell image corresponding to the dwell time from the dwell based on the dwell time information when receiving a command to perform an image summary; And
A summary image is generated by moving the objects that existed in different time zones in the ROI to be located at the same time zone using the object stay image to generate a summary image. Image summarization process that controls only the entry time or departure time for each object to be continuously displayed or the remaining time of the object is changed to be displayed as time passes.
A computer-readable recording medium having recorded thereon a program for realizing this. The method of claim 16,
The video summary process,
Determining that the object has entered the ROI when the object is detected in the ROI based on the dwell time information and performing the image summary; And
And determining that the object has left the region of interest when the object is not detected in the region of interest, and stopping the summary of the image. The method of claim 16,
When a plurality of ROIs are set, the method may further include setting importance of the plurality of ROIs.
The image summarizing process is a computer-readable recording medium having recorded thereon a program for performing the image summarization based on the dwell time information of the region of interest corresponding to the particular importance level when receiving selection information on a particular importance level.

Images