KR20220129943A - Apparatus and method for generating a panoramic video image by combining multiple video images - Google Patents

Abstract

The present invention relates to a device and a method for generating a panoramic video image by combining a plurality of video images, which can provide a more convenient and efficient working environment. The device of the present invention comprises: a video camera unit including a plurality of video cameras; a memory for receiving and storing the video images; a display unit for displaying a graphical user interface (GUI); and a processor for generating one panoramic video image.

Description

Apparatus and method for generating a panoramic video image by combining a plurality of video images

The present disclosure relates to the processing of video images received by a video surveillance system, and more particularly, to a panoramic video image by combining multiple video images for subsequent analysis of a plurality of video images or portions thereof acquired through a plurality of cameras. It is about technology to create

In general, a video surveillance system refers to a system for monitoring a specific location by installing one or more cameras at a specific location and displaying an image acquired through a camera installed at a specific location at another location. This video surveillance system stores video data based on the characteristics of the video information, so that when an incident occurs, it is possible to quickly search for the situation and efficiently manage the data. For example, the video surveillance system digitizes the information of the video camera, analyzes the location and pattern of objects according to the pattern analysis defined in advance, and when an intrusion or accident occurs, an alarm, blocking system operation, and notification information are provided to the security officer can do.

Today, the demand for the establishment of a video surveillance system is increasing due to the increased awareness of the security industry and customers, and the application of the video surveillance system is expanding in order to prevent surveillance due to the recent increase in fires and violent crimes in Korea.

At the same time, the image detection technology is demanding the development of more advanced technologies due to the development and market expansion of CCTV cameras and digital storage device image analysis and recognition technologies.

However, when monitoring (or monitoring) a wide area using such an image monitoring system, image data of each area is collected through a plurality of cameras and displayed through a neighboring layout, but the situation of the entire area is monitored. It is difficult to observe. For example, the airport surveillance system may secure a view of the entire area through a plurality of cameras. In this case, each camera is provided to have a field of view overlapping each other, so that image data for the entire area may be acquired. However, it is difficult to examine a specific action or situation by comparing the image data collected from each camera in a wide tracking area. For example, the probability of a recognition error may increase in the image analysis process, and fatigue of an operator who wants to compare each image may be improved. That is, there is a risk that the monitoring fatigue of the operator is accumulated due to the lack of a consistent field of view, or the object and situation detection efficiency is lowered as the response time to a specific event is delayed.

Therefore, there may be a need in the industry for image sensing technology to reduce the possibility of recognition errors and accurately analyze events by combining individual video streams obtained from each camera into a panoramic video for more efficient monitoring.

Laid-open Patent Publication No. 10-2012-0114050 (2012.10.16)

SUMMARY OF THE INVENTION An object of the present disclosure is to solve the above-described problem, and to generate a panoramic video image by combining a plurality of video images. In addition, the present disclosure may provide a more convenient and efficient working environment by providing a video surveillance system through a graphic user interface including a specific block having a specific function based on a combination/panoramic video image.

The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

Disclosed is a computing device for combining a plurality of video images according to various embodiments of the present disclosure for solving the above problems. The computing device includes a video camera unit including a plurality of video cameras for acquiring the plurality of video images, a memory for receiving and storing the plurality of video images from the video camera unit, and tools for input and output a display unit for displaying a graphical user interface (GUI) including a selection block for receiving the client's selection input for at least two or more specific adjacent video cameras, a region of interest for receiving the client's selection input for an arbitrary region to present a video image separated within the panoramic video image a selection block and a display block for displaying the plurality of video images or the panoramic video images, wherein the processor is configured to receive a plurality of video images from each of the two or more specific adjacent video cameras selected through the selection block; setting an initial transformation necessary to combine the plurality of video images into the panoramic video image, and performing the initial transformation on video images temporally corresponding to each other among the video images of each of the adjacent video cameras; The plurality of video images are combined into a panoramic video image by performing subsequent conversion on other images in addition to the video images on which the conversion has been performed, and the panoramic video image is displayed in one video camera layout cell included in the display block. another one included in the display block when receiving a selection input of a client for any region of interest for displaying the separated video image in the panoramic video image through the region of interest selection block The separated above in the video camera layout cell of You may decide to display a video image.

In an alternative embodiment, the processor may perform real-time combining or archive combining of the plurality of video images.

In an alternative embodiment, the plurality of video images may be combined in real time and transmitted directly from the plurality of video cameras to the processor.

In an alternative embodiment, the plurality of video images may be combined via archived data and transferred from the memory to the processor.

In an alternative embodiment, the processor may include a video card module that participates in a resource in the initial conversion setting process and combines the plurality of video images into one panoramic video image.

In an alternative embodiment, the processor performs a search for an optimal local point in each video image synchronously received from the at least two or more specific neighboring cameras, the optimal local point being the at least two It may be a point that is compared for the subsequent transformation for a plurality of video images received from the above specific neighboring cameras.

In an alternative embodiment, the optimal local point may be further determined in response to the input of the client in the passive mode.

In an alternative embodiment, the processor receives lens parameter information from each of the plurality of video cameras, wherein the lens parameter information may be utilized to calculate the initial transformation and the subsequent transformation.

In an alternative embodiment, the at least one of the initial transformation and the subsequent transformation may include at least one of adjusting the brightness of the video image and adjusting the contrast of the video image.

In an alternative embodiment, the at least two or more specific adjacent cameras may have different frame rates from each other.

In an alternative embodiment, the video images received from each of the at least two or more specific adjacent cameras may have different display resolutions.

In an alternative embodiment, the at least two or more specific adjacent cameras may be characterized in that they have an overlapping field of view.

In an alternative embodiment, the processor is configured to: perform successive combining of a pair of temporally corresponding video images among a plurality of video images received from each of the at least two or more specific adjacent cameras, thereby continuously combining the plurality of video images. Images can be combined to create the panoramic video image.

In an alternative embodiment, the processor may be configured to export the panoramic video image or the separated video image corresponding to a region of the panoramic video image to an external device.

In an alternative embodiment, the processor may perform video analysis on the panoramic video image.

In an alternative embodiment, the video analysis includes one or more searches based on metadata, wherein the one or more searches may include a face search, a license plate number search, a moving object search, and an event search.

In an alternative embodiment, the separated video images obtained in response to the client's selection of the arbitrary region of interest are separately stored in another video camera layout cell different from the panoramic video image displayed in the one video camera layout cell. can be displayed simultaneously as an image of

In an alternative embodiment, the panoramic video image may be an image relating to a synthesized circular view viewed from above.

A method of combining a plurality of video images implemented by a computing device including a processor and a memory according to another embodiment of the present disclosure is disclosed. The method includes: receiving a selection input relating to at least two specific adjacent video cameras of a plurality of video cameras; receiving and storing a plurality of video images from each of the at least two or more specific adjacent video cameras; setting an initial transformation necessary for combining the video images of , into a panoramic video image, and performing the initial transformation on video images temporally corresponding to each other among the video images corresponding to each of the adjacent video cameras, the transformation Combining the plurality of video images into the panoramic video image by performing subsequent transformation on other video images in addition to the performed video images, and adding the panoramic video image to one video camera layout cell included in the display block displaying, receiving input from a client for an arbitrary area for displaying a separated video image within the panoramic video image, and the separated video image in another video camera layout cell included in the display block. It may include the step of displaying

In an alternative embodiment, the plurality of video images may be combined in real time or by archived data.

In an alternative embodiment, the plurality of video images may be combined in real time and transmitted directly from the plurality of video cameras to the processor.

In an alternative embodiment, the plurality of video images may be combined via archived data and transferred from the memory to the processor.

In an alternative embodiment, the resource of the video card module included in the processor may be utilized in the step of setting the initial conversion and the step of combining the plurality of video images into one panoramic video.

In an alternative embodiment, the step of setting the initial transformation is characterized in that the search for an optimal local point is performed in each video image synchronously received from at least the at least two or more specific neighboring cameras, The optimal local point may be a point compared for the subsequent transformation of a plurality of video images received from the at least two or more specific adjacent cameras.

In an alternative embodiment, the optimal local point may be additionally obtained in response to the input of the client in the passive mode.

In an alternative embodiment, in the setting step, lens parameter information is received from each of the plurality of video cameras, and the lens parameter information is utilized to calculate the initial transformation and the subsequent transformation. .

In an alternative embodiment, at least one of the initial transformation and the subsequent transformation may include at least one of adjusting a brightness of a video image and adjusting a contrast of a video image.

In an alternative embodiment, the at least two or more specific adjacent cameras may be characterized as having different frame rates from each other.

In an alternative embodiment, the video images received from each of the at least two or more adjacent cameras may have different resolutions.

In an alternative embodiment, the at least two or more specific adjacent cameras may be characterized in that they have an overlapping field of view.

In an alternative embodiment, the step of generating the panoramic video image by combining the plurality of video images may include: pairing temporally corresponding video images among a plurality of images received from each of the at least two or more specific adjacent cameras. It can be carried out through successive combinations of

In an alternative embodiment, the panoramic video image or the separated image corresponding to a region of the panoramic video image may be exported to an external device.

In an alternative embodiment, video analysis on the panoramic video image may be additionally performed.

In an alternative embodiment, the video analysis may include one or more searches based on metadata, wherein the one or more searches may include a face search, a license plate number search, a moving object search, and an event search.

In an alternative embodiment, the separated video images obtained in response to the client's selection of the arbitrary region of interest are separately stored in another video camera layout cell different from the panoramic video image displayed in the one video camera layout cell. can be displayed simultaneously as an image of

In an alternative embodiment, the panoramic video image may be an image relating to a synthesized circular view viewed from above.

According to another embodiment of the present disclosure, a computer program stored in a computer-readable storage medium including instructions executed by one or more processors to implement a technique for combining a plurality of video images is disclosed. One or more processors of the computer program may perform the steps of the method.

Other specific details of the present disclosure are included in the detailed description and drawings.

According to various embodiments of the present disclosure, it is possible to provide a panoramic video image having a consistent field of view based on a plurality of video images obtained from a plurality of cameras, thereby providing an effect of improving the efficiency of monitoring (or detection).

Effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a conceptual diagram illustrating a system in which various aspects of a computing device for generating a panoramic video image by performing a combination on a plurality of video images according to various embodiments of the present disclosure may be implemented.
2 is a block diagram of a computing device for generating a panoramic video image by performing combination on a plurality of video images according to an embodiment of the present disclosure.
3 is a diagram illustrating an exemplary configuration of a graphical user interface related to an embodiment of the present disclosure.
4 shows an exemplary diagram exemplarily illustrating a method of displaying a plurality of video images, a panoramic video image, and a separated video image through one or more video camera layout cells according to an embodiment of the present disclosure.
5 is a flowchart exemplarily illustrating steps of a method for providing a panoramic video image method according to an embodiment of the present disclosure.

Advantages and features of the present disclosure and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments allow the disclosure of the present disclosure to be complete, and those of ordinary skill in the art to which the present disclosure pertains. It is provided to fully understand the scope of the present disclosure to those skilled in the art, and the present disclosure is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present disclosure. In this specification, the singular also includes the plural, unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present disclosure.

Unless otherwise defined, all terms (including technical and scientific terms) used herein may have the meaning commonly understood by those of ordinary skill in the art to which this disclosure belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless clearly specifically defined.

As used herein, the term “unit” or “module” refers to a hardware component such as software, FPGA, or ASIC, and “unit” or “module” performs certain roles. However, “part” or “module” is not meant to be limited to software or hardware. A “part” or “module” may be configured to reside on an addressable storage medium or may be configured to reproduce one or more processors. Thus, by way of example, “part” or “module” refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, Includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. Components and functionality provided within “parts” or “modules” may be combined into a smaller number of components and “parts” or “modules” or additional components and “parts” or “modules”. can be further separated.

In this specification, a computer means all types of hardware devices including at least one processor, and may be understood as encompassing software configurations operating in the corresponding hardware device according to embodiments. For example, a computer may be understood to include, but is not limited to, smart phones, tablet PCs, desktops, notebooks, and user clients and applications running on each device.

Those skilled in the art will further appreciate that the various illustrative logical blocks, configurations, modules, circuits, means, logics, and algorithm steps described in connection with the embodiments disclosed herein may be implemented in electronic hardware, computer software, or combinations of both. It should be recognized that they can be implemented with To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, configurations, means, logics, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application. However, such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

Each step described in this specification is described as being performed by a computer, but the subject of each step is not limited thereto, and at least a portion of each step may be performed in different devices according to embodiments.

1 is a conceptual diagram illustrating a system in which various aspects of a computing device for generating a panoramic video image by performing a combination on a plurality of video images according to various embodiments of the present disclosure may be implemented.

As shown in FIG. 1 , the system may include a computing device 100 , a client terminal 10 , an external server 20 , and a network. The components constituting the above-described system are exemplary, and the scope of the present disclosure is not limited to the above-described components. That is, additional components may be included or some of the above-described components may be omitted depending on implementation aspects for the embodiments of the present disclosure. The computing device 100 , the client terminal 10 , and the external server 20 according to embodiments of the present disclosure may mutually transmit/receive data for the system according to embodiments of the present disclosure through a network.

According to an embodiment of the present disclosure, a network may transmit/receive image data, a plurality of video images, and a panoramic video image according to an embodiment of the present disclosure, to another computing device, a server, and the like. That is, the network may provide a communication function between the system and the client terminal and the external server. For example, the network may transmit a panoramic video image to an external server (eg, a security server), or may receive information about a user's selection input for at least two specific cameras among a plurality of video cameras from a client terminal. Additionally, the network may allow information transfer between the computing device 100 and the client terminal 10 or the external server 20 in a manner that calls a procedure to the computing device 100 .

Networks according to embodiments of the present disclosure include Public Switched Telephone Network (PSTN), x Digital Subscriber Line (xDSL), Rate Adaptive DSL (RADSL), Multi Rate DSL (MDSL), Very High Speed DSL (VDSL). ), UADSL (Universal Asymmetric DSL), HDSL (High Bit Rate DSL), and various wired communication systems such as local area network (LAN) can be used.

In addition, the networks presented here are Code Division Multi Access (CDMA), Time Division Multi Access (TDMA), Frequency Division Multi Access (FDMA), Orthogonal Frequency Division Multi Access (OFDMA), Single Carrier-FDMA (SC-FDMA) and Various wireless communication systems may be used, such as other systems.

The network according to the embodiments of the present disclosure may be configured regardless of its communication mode, such as wired and wireless, and is composed of various communication networks such as a personal area network (PAN) and a wide area network (WAN). can be In addition, the network may be a well-known World Wide Web (WWW), and may use a wireless transmission technology used for short-range communication such as Infrared Data Association (IrDA) or Bluetooth. The techniques described herein may be used in the networks mentioned above, as well as in other networks.

The computing device 100 of the present disclosure may be a computing device that provides information through a network to a terminal related to a client that wants to monitor a specific area. The computing device 100 may acquire a plurality of video images, and may generate a panoramic video image based on the plurality of acquired video images. In this case, the plurality of video images may be video images acquired through each of the plurality of video cameras at different angles. Specifically, the computing device 100 may generate a panoramic video image by combining a plurality of video images obtained from respective video cameras having different fields of view. In this case, each of the plurality of video cameras may be provided to acquire a video image of an area partially overlapped with an adjacent video camera. The panoramic video image may be a video image for simultaneously capturing an angle of view greater than the viewing angle viewed by the user through a plurality of cameras, and securing a wider viewing angle (eg, 360-degree field of view) by combining the captured images. For example, the panoramic video image may be a video image related to a synthesized circular view viewed from above. That is, the computing device 100 may provide a wider field of view in relation to various areas by generating a panoramic video image based on video images acquired through a plurality of cameras. Accordingly, the client can more easily grasp the situation in the corresponding area through one panoramic video image, without the need to perform comparison between video images corresponding to each camera in order to grasp the situation of the specific area. can be provided

Also, the computing device 100 may perform video analysis on the generated panoramic video image. Video analysis may include one or more searches based on metadata. The one or more searches refer to a search related to an object existing in a video image, and may include, for example, a face search, a vehicle registration number search, a specific object search, a motion search of an object, a specific event search, and the like. However, the video analysis performed by the computing device 100 of the present disclosure is not limited thereto, and may further include various types of analysis related to a video image.

For example, for monitoring a wide area, a plurality of cameras for observing a corresponding area may be provided, and a plurality of video images may be acquired through each of the plurality of cameras. In this case, video analysis may be performed on a plurality of video images of the corresponding area for monitoring. However, when video analysis is performed corresponding to each of a plurality of acquired video images, as each video image has a different field of view in each of a wide area, there is a fear that a recognition error of the video analysis may increase. That is, the computing device 100 of the present disclosure generates a panoramic video image having a consistent field of view by combining a plurality of video images having different fields of view in order to solve such a concern, thereby increasing the efficiency and efficiency in the video analysis process. performance can be improved.

Also, the computing device 100 may generate a graphical user interface according to embodiments of the present disclosure. The graphical user interface may include one or more video camera layout cells. Each of the one or more video camera layout cells may be a divided area for displaying various video images. The various video images may include, for example, at least one of a plurality of video images, a panoramic video image generated based on the plurality of video images, and a separate video image relating to a specific region of interest in the panoramic video image. The video image displayed in each of the one or more video camera layout cells may be a video image associated with the client's selection. For example, a first video layout cell of the one or more video layout cells may display first video images acquired through a first video camera, and the second video layout cell may be a second video camera adjacent to the first video camera. may display the second video images obtained through , and the third video layout cell may display panoramic video images generated through the combination of the first video images and the second video images. The detailed description of the video images displayed in each of the one or more video layout cells described above is only an example, and the present disclosure is not limited thereto.

That is, the graphical user interface may enable the client to monitor images of various fields of view corresponding to a specific area by simultaneously displaying various video images related to the selection of the client through one or more video layout cells. Accordingly, the monitoring efficiency of a client monitoring a specific area may be improved.

Also, the computing device 100 may transmit the generated graphic user interface to the client terminal 10 . In this case, the client terminal 10 may be any type of computing device capable of accessing the system. The computing device 100 may transmit a graphical user interface to the client terminal 10 through a network.

The computing device 100 according to an embodiment of the present disclosure may be a terminal or a server, and may include any type of device. The computing device 100 is a digital device, and may be a digital device equipped with a processor and having a computing capability, such as a laptop computer, a notebook computer, a desktop computer, a web pad, a tablet, and a mobile phone. The computing device 100 may be a web server that processes a service. The above-described types of servers are merely examples and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the computing device 100 may be a server that provides a cloud computing service. More specifically, the computing device 100 is a type of Internet-based computing, and may be a server that provides a cloud computing service that processes information not with a user's computer but with another computer connected to the Internet. The cloud computing service may be a service that stores data on the Internet and allows the user to use it anytime and anywhere through Internet access without installing necessary data or programs on his/her computer. It's easy to share and deliver with a click. In addition, the cloud computing service not only stores data on a server on the Internet, but also enables users to perform desired tasks using the functions of applications provided on the web without installing a separate program. It may be a service that allows you to work while sharing. In addition, the cloud computing service may be implemented in the form of at least one of Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Software as a Service (SaaS), a virtual machine-based cloud server, and a container-based cloud server. . That is, the computing device 100 of the present disclosure may be implemented in the form of at least one of the above-described cloud computing services. The detailed description of the above-described cloud computing service is merely an example, and may include any platform for building the cloud computing environment of the present disclosure.

According to an embodiment of the present disclosure, the client terminal 10 is a terminal capable of receiving monitoring information related to a specific area through information exchange with the computing device 100 , and may refer to a terminal possessed by the client. For example, the client terminal 10 may be a terminal related to a user who wants to detect various events that may occur in a specific area. The client may receive, through the client terminal 10 , information on a panoramic video image related to a specific region and notification information that a specific event has occurred in the corresponding panoramic video image.

The client terminal 10 is a customer terminal (UE), a mobile, a PC capable of wireless communication, a mobile phone, a kiosk, a cellular phone, a cellular, a cellular terminal, a subscriber unit, a subscriber station, a mobile station, a terminal, a remote station, a PDA, a remote terminal Any that can use a wireless mechanism, such as an access terminal, user agent, cellular phone, wireless phone, session initiation protocol (SIP) phone, wireless local loop (WLL) station, portable device with wireless access capability, wireless model, etc. It may be referred to as a device of a, but is not limited thereto. In addition, the client terminal 10 may be referred to as any device capable of using a wired connection mechanism, such as a wired fax machine, a PC equipped with a wired model, a wired telephone, a terminal capable of wired communication, etc., but is not limited thereto. does not

According to an embodiment of the present disclosure, the external server 20 may refer to an external storage device connected to the computing device 100 in order to easily secure insufficient storage space in the computing device 100 and the client environment. The external server 20 may expand the storage space of data by adding an external storage device to the computing device 100 connected to the network. For example, the external server 20 may receive and store at least some of the plurality of video images acquired by the computing device 100 (eg, video images obtained by a predetermined time elapsed). The external server 20 is a digital device, and may be a digital device equipped with a processor, such as a laptop computer, a notebook computer, a desktop computer, a web pad, and a mobile phone, and having a computing capability with a memory. The external server 20 may be a web server that processes a service. The above-described types of external servers are merely examples, and the present disclosure is not limited thereto.

Hereinafter, a method for the computing device 100 to generate a panoramic video image by combining a plurality of video images with reference to FIG. 2 and a graphic user interface for displaying the generated panoramic video image will be described in more detail below.

2 is a block diagram of a computing device for generating a panoramic video image by performing combination on a plurality of video images according to an embodiment of the present disclosure.

As shown in FIG. 2 , the computing device 100 may include a network unit 110 , a display unit 130 , an input unit 140 , a memory 150 , and a processor 160 . Components included in the aforementioned computing device 100 are exemplary and the scope of the present disclosure is not limited to the aforementioned components. That is, additional components may be included or some of the above-described components may be omitted depending on implementation aspects for the embodiments of the present disclosure.

According to an embodiment of the present disclosure, the computing device 100 may include a network unit 110 that transmits and receives data between the client terminal 10 and the external server 20 . In an embodiment, the network unit 110 may receive information input from an external device or application other than the above-described component. Examples of the external device may include a PC, a portable terminal, a wearable device, and the like. Also, the network unit 110 may form a channel for data communication between the video camera unit 120 , the display unit 130 , the input unit 140 , and the processor 160 .

The network unit 110 according to an embodiment of the present disclosure includes a Public Switched Telephone Network (PSTN), x Digital Subscriber Line (xDSL), Rate Adaptive DSL (RADSL), Multi Rate DSL (MDSL), VDSL ( A variety of wired communication systems such as Very High Speed DSL), Universal Asymmetric DSL (UADSL), High Bit Rate DSL (HDSL), and Local Area Network (LAN) can be used.

In addition, the network unit 110 presented herein is CDMA (Code Division Multi Access), TDMA (Time Division Multi Access), FDMA (Frequency Division Multi Access), OFDMA (Orthogonal Frequency Division Multi Access), SC-FDMA ( A variety of wireless communication systems may be used, such as Single Carrier-FDMA) and other systems.

In the present disclosure, the network unit 110 may be configured regardless of its communication mode, such as wired and wireless, and may be configured with various communication networks such as a personal area network (PAN) and a wide area network (WAN). can In addition, the network unit 110 may be a known World Wide Web (WWW), using a wireless transmission technology used for short-range communication such as infrared (IrDA) or Bluetooth (Bluetooth). may be The techniques described herein may be used in the networks mentioned above, as well as in other networks.

The video camera unit 120 of the present disclosure may include a plurality of cameras for acquiring a video image related to a specific place. For example, the video camera unit 120 may include a plurality of video cameras provided in a direction or a photographing angle corresponding to a specific area in order to acquire a plurality of video images for monitoring the specific area. In this case, each of the plurality of video cameras may be provided to have a partially overlapping field of view at each of different positions in order to photograph (or monitor) a specific area. For example, the first video images and the second video images acquired through each of the first and second video cameras adjacent to each other may include at least a partial overlapping area with each other. The overlapping area may mean an area equally included in the first video image and the second video image. The plurality of video images may be provided to have a high field of view in order to more easily detect an event that may occur in a specific area.

According to another embodiment of the present disclosure, each of the specific adjacent video cameras may have a different frame rate from each other. The frame rate may mean a speed at which one video image is displayed. For example, the frame rates of the first video camera and the second video camera adjacent to each other may be 12 fps and 25 fps, respectively. In other words, the first video images acquired through the first video camera may display 12 frames per second, and the second video images acquired through the second video camera may display 25 frames per second. As the frame rates of each of the adjacent cameras are different, efficiency in the process of combining a plurality of images received through each camera may be improved.

According to another embodiment of the present disclosure, each of the specific adjacent video cameras may have different display resolutions. The resolution may mean the quality of the video image. As the resolutions of adjacent cameras are different from each other, efficiency in the process of combining a plurality of images received through each camera may be improved. In this case, in a subsequent combining process, the resolutions of all video images may be converted to match each other.

According to an embodiment of the present disclosure, the display unit 130 may output any type of information generated or determined by the processor 160 and any type of information received through a network.

According to an embodiment of the present disclosure, the display unit 130 may display a plurality of video images acquired through the video camera unit 120 . For example, the display unit 130 may display a plurality of images obtained from a specific video camera among a plurality of video cameras included in the video camera unit 120 . In this case, the plurality of displayed images is a combination of images acquired for each viewpoint, and may mean that each video image is arranged and visualized according to the passage of time. That is, the display unit 130 may receive and display a plurality of video images from each of the plurality of video cameras, thereby enabling the user to monitor an area photographed by each video camera.

Also, the display unit 130 may display a graphic user interface including tools related to input and output. The display unit 130 may display a graphic user interface as shown in FIG. 3 . The detailed description of the graphical user interface shown and described below is merely an example, and the present disclosure is not limited thereto.

In an embodiment of the present disclosure, the display unit 130 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode, OLED), a flexible display, and a three-dimensional display (3D display) may include at least one. Some of these display modules may be configured as a transparent type or a light transmitting type so that the outside can be viewed through them. This may be referred to as a transparent display module, and a representative example of the transparent display module is a transparent OLED (TOLED).

In embodiments of the present disclosure, “display” may be an operation of displaying data to a user through the display unit 130 . “Display” and “display” may be used interchangeably.

According to an embodiment of the present disclosure, the computing device 100 may include an input unit 140 for accepting an input from a client. The input unit 140 may receive various inputs from the client. For example, the input unit 140 may receive a selection input of a client for displaying a video image corresponding to a specific camera or a selection input of a client related to a specific region of interest in the panoramic video image. According to a client input through the input unit 140 , a computer program for providing a panoramic video image based on a plurality of video images according to embodiments of the present disclosure may be executed.

The input unit 140 according to embodiments of the present disclosure may receive a signal by sensing a button manipulation or a touch input of a client, or may receive a user's voice or motion through a camera or a microphone and convert it into an input signal. . For this, speech recognition technology or motion recognition technology may be used.

The input unit 140 according to embodiments of the present disclosure may be implemented as an external input device connected to the computing device 100 . For example, the input device may be at least one of a mouse, a keyboard, a touchpad, a joystick, a trackpad, a stylus, an input button, a touch screen, or a combination thereof, but this is only an example and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the input unit 140 may have the same configuration as the display unit 130 . For example, the input unit 140 may be configured as a touch screen configured to receive a selection input from a client. For the touch screen, any one of a contact capacitive method, an infrared light sensing method, a surface ultrasonic wave (SAW) method, a piezoelectric method, and a resistive film method may be used. The detailed description of the above-described touch screen is merely an example according to an embodiment of the present invention, and various touch screen panels may be employed in the computing device 100 . The input unit 140 configured as a touch screen may include a touch sensor. The touch sensor may be configured to convert a change in pressure applied to a specific portion of the input unit 140 or capacitance generated at a specific portion of the input unit 140 into an electrical input signal. The touch sensor may be configured to detect not only the touched position and area, but also the pressure at the time of the touch. When there is a touch input to the touch sensor, a signal(s) corresponding thereto is sent to the touch controller. The touch controller processes the signal(s) and then sends the corresponding data to the processor 160 . Accordingly, the processor can recognize which area of the input unit 140 has been touched, and the like. Accordingly, the display unit 130 of the present disclosure may allow input/output for information exchange between the client and the computing device 100 . Accordingly, the client can access all visible screen objects (interface devices) displayed (or screened) through the display unit 130 .

According to an embodiment of the present disclosure, the memory 150 may store any type of information generated or determined by the computing device 100 and any type of information received through a network. Also, the memory 150 may store information about a plurality of video images received from the video camera unit 120 . Specifically, the memory 150 may archive video images received from each of all video cameras included in the video camera unit 120 . The archive may mean that real-time information (eg, information about a plurality of video images acquired in real-time) is generated and stored at the same time. Such archives can maximize the speed of information production, delivery and sharing and improve access to knowledge. For example, the memory 150 may temporarily or permanently store input/output data (eg, information on a plurality of video images and panoramic video images).

In addition, the memory 150 may store a computer program for performing a method for generating a panoramic video image through a combination of a plurality of video images according to an embodiment of the present disclosure, and the stored computer program uses the computing device. It may be read and driven by one or more processors constituting it.

According to an embodiment of the present disclosure, the memory 150 is a flash memory type, a hard disk type, a multimedia card micro type, or a card type memory (eg, For example, SD or XD memory), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read (PROM) -Only Memory), a magnetic memory, a magnetic disk, and an optical disk may include at least one type of storage medium. The computing device 100 may operate in relation to a web storage that performs a storage function of the memory 150 on the Internet. The description of the above-described memory is only an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 160 may read a computer program stored in the memory 150 and provide a panoramic video image according to an exemplary embodiment of the present disclosure.

According to an embodiment of the present disclosure, the processor 160 may be configured with one or more cores, and may include a central processing unit (CPU) and a general purpose graphics processing unit (GPGPU) of a computing device. , data analysis such as a tensor processing unit (TPU), and a processor for deep learning. The processor 160 may typically process the overall operation of the computing device 100 .

The processor 160 processes signals, data, information, etc. input or output through the above-described components, or by driving an application program stored in the memory 150, to provide or process appropriate information or functions to the client. have.

According to an embodiment of the present disclosure, the processor 160 may generate a panoramic video image based on a plurality of video images. The processor 160 processes signals, data, information, etc. input or output through the above-described components, or by driving an application program for providing a graphical user interface, thereby providing or processing appropriate information or functions to the client. have.

Specifically, the processor 160 may generate a panoramic video image by performing real-time combining or archive combining on a plurality of video images. The processor 160 may receive a plurality of video images from at least one of a plurality of video cameras and the memory 150 included in the video camera unit 120, and perform a combination on the plurality of received video images. You can create panoramic video images. The panoramic video image may be a video image for simultaneously capturing an angle of view greater than the viewing angle viewed by the user through a plurality of cameras, and securing a wider viewing angle (eg, 360-degree field of view) by combining the captured images. The processor 160 may include a video card module. The video card module may participate in a resource in the initial conversion setting process, and may perform a combination of a plurality of video images into one panoramic video image. For example, the video card module included in the processor 160 is configured as a graphics processing unit, so that the panoramic video image generation process can be accelerated.

As a specific example, when receiving a plurality of video images directly from the video camera unit 120 , the processor 160 may generate a panoramic video image through real-time combination of the plurality of images. That is, the plurality of video images may be directly transmitted from the plurality of video cameras to the processor 160 to be combined in real time.

In addition, when receiving a plurality of archived video images from the memory 150 , the processor 160 may generate a panoramic video image through an archive combination of the plurality of archived video images. That is, the plurality of video images may be transmitted from the memory 150 to the processor 160 to be combined based on the archived data. When a plurality of video images archived in the memory 150 are utilized, desired video images may be acquired as necessary to generate a panoramic video image, so that clearer monitoring information may be obtained in various situations.

The processor 160 may generate a panoramic video image of the present disclosure based on at least one of a plurality of images acquired in real time through the video camera unit 120 and a plurality of archived images stored in the memory 150 . have. In other words, the processor 160 may generate a panoramic video image in relation to a real-time image, or may generate a panoramic video image in relation to an image of a past viewpoint (ie, a plurality of archived video images). Accordingly, the processor 160 may generate and provide a panoramic video image in relation to various viewpoints.

According to an embodiment of the present disclosure, the processor 160 may generate a graphic user interface and display the generated graphic user interface 200 through the display unit 130 . The graphical user interface 200 is based on displaying all system objects and functions available to the user in the form of graphical screen components (windows, icons, menus, buttons, lists, etc.) between the computing device 100 and the client. It may be a user interface for interaction.

The graphical user interface generated by the processor 160 may include a selection block 210 , a region of interest selection block 220 , and a display block 230 as shown in FIG. 3 .

The selection block 210 included in the graphical user interface 200 may be a block for receiving a selection input from a client for at least two adjacent video cameras among a plurality of video cameras. Specifically, the selection block 210 may be a block for receiving a selection of a client for at least two or more adjacent video cameras in order to perform a combination on a plurality of video images having different fields of view. That is, when the client selects at least two video cameras through the selection block 210, the processor 160 may generate a panoramic video image by performing a combination between a plurality of images corresponding to the two or more video cameras. .

For example, the video camera unit 120 includes a first video camera, a second video camera, a third video camera, and a fourth video camera, and through the selection block 210 , the client selects a second video camera adjacent to each other and When the third video camera is selected, the processor 160 combines the second video images and the third video images acquired through each of the second video camera and the third video camera based on the selection of the corresponding client to obtain a panoramic video image. can create The specific description of the above-described video camera and video image is merely exemplary, and the present disclosure is not limited thereto.

That is, the graphical user interface 200 may include a selection block 210 for receiving a client's input for a video image on which the panoramic video image is generated. Accordingly, the client may determine an area (ie, an area photographed by video cameras) that the client wants to monitor through a wider field of view through the selection block 210 .

Also, the region of interest selection block 220 included in the graphical user interface 200 may be a block for receiving a selection input from a client for an arbitrary region in order to display a video image separated within the panoramic video image. The separated video image may be a video image obtained by extracting only a specific region within the panoramic video in order to more easily identify a specific region of the panoramic video image. Specifically, the region of interest selection block 220 may receive a selection input from the client regarding an arbitrary region for more closely monitoring within a panoramic video image that is a combination of a plurality of video images. For example, the region of interest selection block 220 may allow a client's drag input for a certain region within the panoramic image, and the processor 160 generates a separated video image by extracting only the region corresponding to the drag input. can do. That is, the client may be provided with only a region that the client wants to examine more closely among the panoramic video images through the region of interest selection block 220 , thereby increasing monitoring efficiency. In addition, the performance range of video analysis may be reduced, which may contribute to improving the accuracy of analysis.

Also, the display block 230 included in the graphic user interface 200 may be a block for displaying a plurality of video images or a panoramic video image. Specifically, the display block 230 may include one or more video camera layout cells. Each of the one or more video camera layout cells may be a divided area for displaying various video images. The various video images may include, for example, at least one of a plurality of video images, a panoramic video image generated based on the plurality of video images, and a separate video image relating to a specific region of interest in the panoramic video image. The video image displayed in each of the one or more video camera layout cells may be a video image associated with the client's selection. For example, a first layout cell of the one or more video camera layout cells may display first video images acquired through a first video camera, and a second layout cell may display a second video camera adjacent to the first video camera. may display second video images obtained through, and the third layout cell may display panoramic video images generated through the combination of the first video images and the second video images, and the fourth layout cell may include: Separate video images extracted from the panoramic video image may be displayed. The detailed description of video images displayed in each of the one or more video camera layout cells described above is merely an example, and the present disclosure is not limited thereto.

That is, the display block 230 displays various video images related to the client's selection through one or more video camera layout cells in a divided area at the same time, so that the client monitors images of various fields of view corresponding to the specific area. can do. Accordingly, the client can monitor a specific area through various fields of view, thereby improving monitoring efficiency.

According to an embodiment of the present disclosure, the processor 160 may generate a panoramic video image by combining a plurality of images obtained from at least two or more adjacent cameras. Specifically, the processor 160 may receive a selection input of a client related to selection of at least two or more adjacent cameras among a plurality of cameras through the graphical user interface 200 , and select cameras corresponding to the selection input of the client. A panoramic video image can be created by combining the video images each acquired. That is, the processor 160 receives video images from each of the cameras selected (or designated) from the client through the graphical user interface 200, and performs a combination between the plurality of received video images to generate a panoramic video image. can

Combinations between video images in the present disclosure may include combinations through all directions, such as right, left, top, bottom, horizontal, and vertical. Therefore, the panoramic video image generated based on the combination between the video images of the present disclosure provides a sense of realism and immersion for the entire area to be detected, and provides a natural and realistic perception at the client to provide a natural and realistic perception of an abnormal situation (eg, theft). , alarm, disaster, etc.) can improve the efficiency of monitoring work.

More specifically, the processor 160 may calculate a transform required to combine the plurality of received video images into a panoramic video image. The processor 160 may convert video images temporally corresponding to each other among video images of each of the adjacent cameras. For example, video images temporally corresponding to each other may mean an ordered pair of video images received through each camera at the same time.

Transformation in the present disclosure may include at least one of adjusting the brightness of the video image and adjusting the contrast of the video image. That is, at least one of the initial transformation and the subsequent transformation may include at least one of adjusting the brightness of the video image and adjusting the contrast of the video image.

For example, the first video image and the second video image acquired through each of the first video camera and the second video camera may have different views, and thus at least one of brightness and contrast may be partially different. Accordingly, the panoramic video image generated through the combination of the first video image and the second video image may be unnatural due to differences in brightness and contrast of each region. Accordingly, the processor 160 may calculate a transformation that adjusts the brightness and contrast of each video image in the process of performing a combination between video images acquired through each video camera to generate a more natural panoramic video image. .

The processor 160 may identify an optimal local point in each video image acquired through each camera to calculate a transform. The optimal local point may be a reference region for combining each of a plurality of video images received from at least two or more adjacent cameras. An optimal local point may be identified based on background complexity of, for example, gradient differences or integral characteristics. The integral characteristic may mean a complex and accumulated characteristic (eg, brightness, contrast, bidirectional gradient drop value, etc.) around a point. For example, the optimal local point may be any location where the same object is identified in some overlapping field of view of each video module. In a further embodiment, the processor 160 may receive input for the best local point from the client via a passive mode. In this case, the optimal local point may be a point compared corresponding to the input of the client in the passive mode. For example, a manual mode may be utilized when it is difficult to identify an optimal local point in the image.

That is, the processor 160 identifies an optimal local point that is a reference region for a combination from each video image received from each of the specific adjacent cameras, and based on the identified optimal local point from each video image, each video image can be combined into a panoramic video image by applying the average of the pixels of Accordingly, in the panoramic video image generated through the combination, the image data that is the basis for generation, that is, the transition between each frame may be smooth. In other words, a more natural panoramic video image may be created by naturally connecting the boundary portion in the process of combining image data.

Also, the processor 160 may combine a plurality of video images into a panoramic video image by applying the same transformation to other images other than the transformed video images. In other words, the transformation set in the previous transformation step may be identically applied to the video images acquired through each video camera at a different time point (eg, a subsequent viewpoint) from the video images on which the initial transformation has been performed.

That is, the processor 160 calculates an initial transformation for a video image received from each of the video modules adjacent to each other at a corresponding viewpoint, and applies (ie, subsequent transformation) to video images subsequently obtained through the corresponding transformation. , can create panoramic video images. Accordingly, computing power may be reduced in a process of generating a panoramic video image through a combination of a plurality of video images, and thus efficiency may be improved.

According to an embodiment of the present disclosure, the processor 160 may receive lens parameter information corresponding to each of a plurality of video cameras. The lens parameter information is information related to the image acquisition performance of each video camera, and may include, for example, information on saturation, contrast, sharpness (or resolution), brightness, and the like. Such lens parameter information may facilitate the search for an optimal local point in each video image in the process of calculating a transform, thereby enhancing the generation efficiency of the parametric video image.

According to an embodiment of the present disclosure, when the processor 160 generates a panoramic video image through a combination of a plurality of video images, it is configured to display in at least one of one or more video camera layout cells included in the graphical user interface. can decide

According to an embodiment of the present disclosure, the processor 160 may perform video analysis on a panoramic video image. Video analysis may include at least one search by metadata. The one or more searches may include a face search, a vehicle registration number search, a moving object search, and an event search.

For example, the computing device 100 of the present disclosure may store an archive of video images acquired through each of a plurality of video cameras included in the video camera unit in the memory 150 in real time. In the process of uploading the video image to the memory 150 or transmitting the video image to the processor 160 , all the video images may form metadata characterizing data on all objects in the image. In this case, the metadata may include detailed description information on all objects (eg, movement trajectories, face-matched identifiers, recognized license plates, etc.) moving in the field of view of each camera. Accordingly, the processor 160 may perform one or more searches for video analysis based on the metadata. That is, the metadata may be utilized for a search purpose within a video image.

For example, when the client has an image (eg, a photo) of a user to be detected, a face search based on a plurality of video images may be performed by generating metadata based on the image. As another example, if there is a registration number of a vehicle that the client wants to detect, by generating metadata based on the number, a vehicle search based on a plurality of video images may be performed.

That is, the processor 160 of the present disclosure reduces the search time by performing video analysis including face search, vehicle registration number search, moving object search, and event search within the panoramic video image, so that the monitoring process of the client It can provide convenience. In addition, as the video image subject to video analysis is a panoramic video image generated through a combination of individual streams, a wider field of view and a uniform field of view can be provided, so that the accuracy and efficiency of video analysis can be improved. .

According to an embodiment of the present disclosure, the processor 160 may generate a separated video image. Specifically, the processor 160 may receive a selection input from the client regarding an arbitrary region of interest in the panoramic video image through the region of interest selection block included in the graphical user interface 200 , and in response to the selection input, the processor 160 may receive a panorama. You can create a separate video image within a video image. That is, the separated video image generated by the processor 160 may be a video image obtained by magnifying a specific area in the panoramic video image. The separated video image may be displayed and displayed on the graphical user interface 200 by the processor 160 . In this case, the separated video image generated in response to the client's selection of an arbitrary region of interest is one video camera layout cell in which a panoramic video image is displayed among one or more video camera layout cells included in the graphical user interface 200 . It can be displayed simultaneously as a separate image in another video camera layout cell different from .

Accordingly, the client may be provided with an enlarged video image (ie, a separated video image) of a region (eg, an ROI) that the client wants to monitor through a wider field of view among the entire region. This can improve the efficiency of monitoring tasks for clients who want to closely examine specific areas within the video image. Additionally, by simultaneously displaying a panoramic video image and a separate video image related to magnification of an area of the panoramic video image through each of a plurality of cells included in the graphical user interface, a method for allowing the client to understand the situation of the corresponding area More diverse monitoring information can be provided.

According to an embodiment of the present disclosure, the processor 160 may determine to transmit a separated video image generated in response to a selection input of a client related to an arbitrary ROI in the panoramic video image to an external device. In the present disclosure, a panoramic video image or a separate image generated through a combination between a plurality of video images may not be archived in the memory 150 . In other words, a plurality of video images (ie, data corresponding to the original) acquired through each of the plurality of video cameras are archived and stored through the memory 150 , while a panoramic video image generated based on each video image is stored. Alternatively, the separated video image corresponding to one region of the panoramic video image may not be archived in the memory 150 . That is, the processor 160 may determine to convert the panoramic video image or the separated video image generated through a combination of a plurality of video images to an arbitrary scale and transmit the converted video image to an external device. For example, the panoramic video image or the separated video image may be converted into a standard file format such as Adult Audio Video Interleave (AVI) or Matroska Multimedia Container (MKV) format and transmitted to an external device. Accordingly, the memory 150 of the present disclosure archives only data related to the original (ie, a plurality of video images), thereby preventing overload due to capacity.

4 shows an exemplary diagram exemplarily illustrating a method of displaying a plurality of video images, a panoramic video image, and a separated video image through one or more video camera layout cells according to an embodiment of the present disclosure. Among the features of the content shown in FIG. 4, for features overlapping with the features described above with respect to FIGS. 1 to 3, refer to the content described in FIGS. 1 to 3 and the description thereof will be omitted herein.

The graphical user interface 200 of the present disclosure includes a selection block 210 , a region of interest selection block 220 , and a display block 230 for allowing input/output for information exchange between the computing device 100 and the client. can do. The selection block 210 and the region-of-interest selection block 220 may be utilized to receive a selection input of a client, and the display block 230 may display data corresponding to the selection input of the client (eg, a plurality of video images, a panorama). It can be utilized to output a video image or a separated video image, etc.).

As shown in FIG. 4 , the display block 230 included in the graphical user interface 200 includes a first video layout cell 211 , a second video layout cell 212 , a third video layout cell 213 and A fourth video layout cell 214 may be included, and each video layout cell may display different types of video images.

As a more specific example, the video camera unit 120 includes a first video camera, a second video camera, a third video camera, and a fourth video camera, and through the selection block 210 , the client selects adjacent first video cameras. When the camera and the second video camera are selected, the processor 160 displays the first video images acquired through the first video camera in response to the selection input of the corresponding client on the first video layout cell 211, and the second It may be determined to display the second video images acquired through the video camera in the second video layout cell 212 .

In addition, the processor 160 may generate a panoramic video image by combining the first video images and the second video images obtained through each of the first video camera and the second video camera. The processor 160 may determine to display the generated panoramic video image in the third video layout cell 213 .

Also, the processor 160 may receive a selection input of the client for an arbitrary region from the panoramic video image displayed in the third video layout cell 213 through the region of interest selection block 220 . For example, an arbitrary region may relate to a region that the user wishes to examine more closely within the panoramic video image. The processor 160 may determine to generate a separated video within the panoramic video image in response to a selection input of the client for an arbitrary region and display the generated video in the fourth video layout cell 214 . That is, the separated video image generated by the processor 160 may be a video image obtained by magnifying a specific area in the panoramic video image.

That is, the graphical user interface provided by the present disclosure can display various angles or various types of video images in each of a plurality of video layout cells through simple interaction with the client, so that the client can conveniently monitor a specific area. can provide In addition, each of the video images acquired through each camera (eg, the first video image and the second video image), a panoramic video image and a panoramic video image generated through a combination of video images to provide a wide and uniform field of view By separately and simultaneously displaying a separate video image related to magnification of an area on each of a plurality of cells, it is possible to provide a variety of monitoring information for the client to understand the situation of the corresponding area. Accordingly, the efficiency of the monitoring operation may be improved. It will be appreciated by those skilled in the art that the descriptions described above with reference to FIG. 4 are merely examples, and that the number of video cameras, a plurality of video images, and layout cells may vary according to implementation aspects of the present disclosure. It will be obvious.

5 shows an exemplary flowchart of a method of providing a panoramic video image in accordance with an embodiment of the present disclosure.

According to an embodiment of the present disclosure, the computing device 100 may receive a selection input regarding at least two specific adjacent video cameras among a plurality of video cameras ( 310 ).

According to an embodiment of the present disclosure, the computing device 100 may receive and store a plurality of video images from each of at least two or more specific adjacent video cameras ( 320 ).

According to an embodiment of the present disclosure, the computing device 100 sets a transformation necessary for combining a plurality of video images into a panoramic video image, and temporally corresponding video images among video images corresponding to each of the adjacent cameras. Transformation may be performed on them ( 330 ).

According to an embodiment of the present disclosure, the computing device 100 may combine a plurality of video images into a panoramic video image by performing transformation on other images in addition to the transformed video images ( 340 ).

According to an embodiment of the present disclosure, the computing device 100 may display a panoramic video image in one video camera layout cell included in the display block ( 350 ).

According to an embodiment of the present disclosure, the computing device 100 may receive a client input for an arbitrary area for displaying a video image separated within the panoramic video image ( 360 ).

According to an embodiment of the present disclosure, the computing device 100 may display the separated video image in another video camera layout cell included in the display block ( 370 ).

The order of the steps illustrated in FIG. 5 described above may be changed if necessary, and at least one or more steps may be omitted or added. That is, the above-described steps are merely embodiments of the present disclosure, and the scope of the present disclosure is not limited thereto.

The steps of a method or algorithm described in relation to an embodiment of the present disclosure may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. A software module may include random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present disclosure pertains.

Components of the present disclosure may be implemented as a program (or application) to be executed in combination with a computer, which is hardware, and stored in a medium. Components of the present disclosure may be implemented as software programming or software components, and similarly, embodiments include various algorithms implemented as data structures, processes, routines, or combinations of other programming constructs, including C, C++ , Java, assembler, etc. may be implemented in a programming or scripting language. Functional aspects may be implemented in an algorithm running on one or more processors.

A person of ordinary skill in the art of the present disclosure will recognize that the various illustrative logical blocks, modules, processors, means, circuits and algorithm steps described in connection with the embodiments disclosed herein include electronic hardware, (convenience For this purpose, it will be understood that it may be implemented by various forms of program or design code (referred to herein as "software") or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. A person skilled in the art of the present disclosure may implement the described functionality in various ways for each specific application, but such implementation decisions should not be interpreted as a departure from the scope of the present disclosure.

The various embodiments presented herein may be implemented as methods, apparatus, or articles of manufacture using standard programming and/or engineering techniques. The term “article of manufacture” includes a computer program, carrier, or media accessible from any computer-readable device. For example, computer-readable media include magnetic storage devices (eg, hard disks, floppy disks, magnetic strips, etc.), optical disks (eg, CDs, DVDs, etc.), smart cards, and flash memory. devices (eg, EEPROMs, cards, sticks, key drives, etc.). Also, various storage media presented herein include one or more devices and/or other machine-readable media for storing information. The term “machine-readable medium” includes, but is not limited to, wireless channels and various other media capable of storing, retaining, and/or carrying instruction(s) and/or data.

It is understood that the specific order or hierarchy of steps in the presented processes is an example of exemplary approaches. Based on design priorities, it is to be understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of the present disclosure. The appended method claims present elements of the various steps in a sample order, but are not meant to be limited to the specific order or hierarchy presented.

The description of the presented embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not intended to be limited to the embodiments presented herein, but is to be construed in the widest scope consistent with the principles and novel features presented herein.

Claims (37)

A computing device for combining a plurality of video images, comprising:
a video camera unit including a plurality of video cameras for acquiring the plurality of video images;
a memory for receiving and storing the plurality of video images from the video camera unit;
a display unit for displaying a graphical user interface (GUI) including tools related to input and output; and
a processor for generating one panoramic video image by combining the plurality of video images;
including,
The tools are
a selection block for receiving a selection input of a client for at least two specific adjacent video cameras of the plurality of video cameras, a selection input of the client for an arbitrary region to present a separate video image within the panoramic video image A region of interest selection block for receiving and a display block for displaying the plurality of video images or the panoramic video images,
The processor is
Receive a plurality of video images from each of the two or more specific adjacent video cameras selected through the selection block, and set an initial transformation necessary for combining the received plurality of video images into the panoramic video image, the adjacent video cameras The initial conversion is performed on video images temporally corresponding to each other among the video images, and subsequent conversion is performed on images other than the video images on which the conversion is performed to convert the plurality of video images into a panoramic video. Combining into an image, determining to display the panoramic video image in one video camera layout cell included in the display block, and displaying the separated video image within the panoramic video image through the region-of-interest selection block determining to display the separated video image in another video camera layout cell included in the display block when receiving a selection input of the client for any region of interest for
A computing device for combining a plurality of video images.
The method of claim 1,
The processor is
performing real-time combination or archive combination of the plurality of video images,
A computing device for combining a plurality of video images.
3. The method of claim 2,
The plurality of video images,
combined in real time and transmitted from the plurality of video cameras directly to the processor;
A computing device for combining a plurality of video images.
3. The method of claim 2,
The plurality of video images,
combined via archived data and transferred from the memory to the processor;
A computing device for combining a plurality of video images.
The method of claim 1,
The processor is
Including a video card module that participates in a resource (resource) in the initial conversion setting process and combines the plurality of video images into one panoramic video image,
A computing device for combining a plurality of video images.
6. The method according to any one of claims 1 to 5,
The processor is
performing a search for an optimal local point in each video image synchronously received from the at least two or more specific neighboring cameras;
The optimal local point is,
a point that is compared for the subsequent transformation of a plurality of video images received from the at least two or more specific adjacent cameras;
A computing device for combining a plurality of video images.
7. The method of claim 6,
The optimal local point is,
Further determined in response to the input of the client in the manual mode,
A computing device for combining a plurality of video images.

7. The method of claim 6,
The processor is
receiving lens parameter information from each of the plurality of video cameras;
The lens parameter information,
utilized to compute the initial transformation and the subsequent transformation;
A computing device for combining a plurality of video images.
6. The method according to any one of claims 1 to 5,
At least one of the initial transformation and the subsequent transformation,
at least one of adjusting the brightness of the video image and adjusting the contrast of the video image;
A computing device for combining a plurality of video images.
The method of claim 1,
The at least two or more specific adjacent cameras,
Characterized in having different frame rates from each other,
A computing device for combining a plurality of video images.
The method of claim 1,
The video image received from each of the at least two or more specific adjacent cameras,
Characterized in having different resolutions (display resolution),
A computing device for combining a plurality of video images.
The method of claim 1,
The at least two or more specific adjacent cameras,
Characterized in having a field of view that overlaps with each other,
A computing device for combining a plurality of video images.
6. The method according to any one of claims 1 to 5,
The processor is
The panoramic video image is generated by combining the plurality of video images by sequentially combining the temporally corresponding video images among the plurality of video images received from each of the at least two or more specific adjacent cameras as a pair. doing,
A computing device for combining a plurality of video images.
The method of claim 1,
The processor is
characterized in that it is possible to export the panoramic video image or the separated video image corresponding to a region of the panoramic video image to an external device,
A computing device for combining a plurality of video images.
The method of claim 1,
The processor is
performing video analysis on the panoramic video image,
A computing device for combining a plurality of video images.
16. The method of claim 15,
The video analysis is
one or more searches based on metadata, wherein the one or more searches include a face search, a vehicle registration number search, a moving object search, and an event search.
A computing device for combining a plurality of video images.
The method of claim 1,
The separated video image obtained in response to the client's selection of the arbitrary region of interest,
The panoramic video image displayed in the one video camera layout cell is displayed simultaneously as a separate image in the other video camera layout cell different from the one displayed in the video camera layout cell,
A computing device for combining a plurality of video images.
The method of claim 1,
The panoramic video image,
An image related to the synthesized circular view seen from above,
A computing device for combining a plurality of video images.
A method of combining a plurality of video images implemented by a computing device comprising a processor and a memory, the method comprising:
receiving a selection input relating to at least two particular adjacent video cameras of the plurality of video cameras;
receiving and storing a plurality of video images from each of the at least two or more specific adjacent video cameras;
Setting an initial transformation necessary for combining the plurality of received video images into a panoramic video image, and performing the initial transformation on video images temporally corresponding to each other among video images corresponding to each of the adjacent video cameras step;
combining the plurality of video images into the panoramic video image by performing subsequent conversion on video images other than the video images on which the conversion has been performed;
displaying the panoramic video image in one video camera layout cell included in a display block;
receiving a client's input for an arbitrary region for displaying a separated video image within the panoramic video image; and
displaying the separated video image in another video camera layout cell included in the display block;
containing,
A method of assembling a plurality of video images implemented by a computing device comprising a processor and a memory.
20. The method of claim 19,
The plurality of video images,
combined by real-time or archived data;
A method of assembling a plurality of video images implemented by a computing device comprising a processor and a memory.
21. The method of claim 20,
The plurality of video images,
combined in real time and transmitted from the plurality of video cameras directly to the processor;
A method of assembling a plurality of video images implemented by a computing device comprising a processor and a memory.
21. The method of claim 20,
The plurality of video images,
combined via archived data and transferred from the memory to the processor;
A method of assembling a plurality of video images implemented by a computing device comprising a processor and a memory.
20. The method of claim 19,
The resource of the video card module included in the processor,
Utilized in the step of setting the initial transformation and combining the plurality of video images into one panoramic video,
A method of assembling a plurality of video images implemented by a computing device comprising a processor and a memory.
24. The method according to any one of claims 19 to 23,
The setting of the initial transformation comprises performing a search for an optimal local point in each video image synchronously received from at least the at least two or more specific adjacent cameras, wherein the optimal local point is , the point being compared for the subsequent transformation for a plurality of video images received from the at least two or more specific adjacent cameras.
A method of assembling a plurality of video images implemented by a computing device comprising a processor and a memory.
25. The method of claim 24,
The optimal local point is,
additionally obtained in response to the input of the client in the manual mode,
A method of assembling a plurality of video images implemented by a computing device comprising a processor and a memory.
25. The method of claim 24,
In the setting step,
receiving lens parameter information from each of the plurality of video cameras;
The lens parameter information,
utilized to compute the initial transformation and the subsequent transformation,
A method of assembling a plurality of video images implemented by a computing device comprising a processor and a memory.
24. The method according to any one of claims 19 to 23,
At least one of the initial transformation and the subsequent transformation,
at least one of adjusting the brightness of the video image and adjusting the contrast of the video image;
A method of assembling a plurality of video images implemented by a computing device comprising a processor and a memory.
20. The method of claim 19,
The at least two or more specific adjacent cameras,
Characterized in having different frame rates from each other,
A method of assembling a plurality of video images implemented by a computing device comprising a processor and a memory.
20. The method of claim 19,
The video image received from each of the at least two or more adjacent cameras,
Characterized in having different resolutions from each other,
A method of assembling a plurality of video images implemented by a computing device comprising a processor and a memory.
20. The method of claim 19,
The at least two or more specific adjacent cameras,
Characterized in having a field of view that overlaps with each other,
A method of assembling a plurality of video images implemented by a computing device comprising a processor and a memory.
24. The method according to any one of claims 19 to 23,
The generating of the panoramic video image by combining the plurality of video images may include: a continuous combination of temporally corresponding video images among a plurality of images received from each of the at least two or more specific adjacent cameras as a pair carried out through
A method of assembling a plurality of video images implemented by a computing device comprising a processor and a memory.
20. The method of claim 19,
The separated image corresponding to the panoramic video image or a region of the panoramic video image is characterized in that it is exported to an external device,
A method of assembling a plurality of video images implemented by a computing device comprising a processor and a memory.
20. The method of claim 19,
Video analysis for the panoramic video image is additionally performed,
A method of assembling a plurality of video images implemented by a computing device comprising a processor and a memory.
34. The method of claim 33,
The video analysis is
one or more searches based on metadata, wherein the one or more searches include a face search, a license plate number search, a moving object search, and an event search.
A method of assembling a plurality of video images implemented by a computing device comprising a processor and a memory.
20. The method of claim 19,
The separated video image obtained in response to the client's selection of the arbitrary region of interest,
The panoramic video image displayed in the one video camera layout cell is displayed simultaneously as a separate image in the other video camera layout cell different from the one displayed in the video camera layout cell,
A method of assembling a plurality of video images implemented by a computing device comprising a processor and a memory.
20. The method of claim 19,
The panoramic video image,
An image related to the synthesized circular view viewed from above,
A method of assembling a plurality of video images implemented by a computing device comprising a processor and a memory.
A computer program stored in a computer readable storage medium comprising instructions executed by one or more processors to implement a technique for combining a plurality of video images according to any one of claims 19 to 36.

Images