KR102537559B1 - Method for providing various ViewMode in Imaging device - Google Patents

Abstract

A method for providing various view modes in an image capture device includes receiving a list of view modes supported by the image capture device in a client terminal, and a media profile conforming to the ONVIF standard from the image capture device in the client terminal. and checking the view mode of the received media profile.

Description

A method for providing various view modes in an imaging device {Method for providing various ViewMode in Imaging device}

The present invention relates to a method of providing various view modes in an NVT conforming to the ONVIF standard.

In the case of a fisheye camera or a multi-sensor camera that has been recently developed, the ViewMode of an image that can be viewed in a client terminal is variable according to the installation position of the camera or the operation of the sensor.

Conventionally, after a client terminal receives an original image captured by a fish-eye lens camera, an output screen is configured by applying a dewarping function as needed in the client terminal.

KR 10-2015-0131322 A

Recently, as the resolution of a fisheye lens camera has increased to support up to 4K video, when a dwarfing function is applied in a client terminal, a problem arises in that the resources of the client terminal increase rapidly.

In order to solve this problem, the present invention intends to provide a method for outputting an image in a desired view mode format in a client terminal when transmitting an image from a fisheye lens and other image processing devices supported by a dewarping function to a client terminal. .

As a preferred embodiment of the present invention, a method for providing various view modes in an image capture device includes a list of view modes supported by the image capture device from an image capture device for which dewarping is supported in a client terminal. receiving; In the client terminal, receiving a media profile conforming to the ONVIF standard from the image capture device and checking a view mode of the received media profile; Supporting an interface capable of newly setting a view mode of a media profile within the received view mode list; further comprising, when the client terminal changes the view mode of the received media profile, the media profile accordingly. Characterized in that the encoding property of is changed.

As a preferred embodiment of the present invention, in the checking step, the view mode information is included in video source configuration information provided by the imaging device.

As a preferred embodiment of the present invention, when digital PTZ (Pent, Tilt, Zoom) is executed in the client terminal, the video recording device performs dewarping according to the view mode in units of media profiles, and then executes digital PTZ. Transmitting one image to the client terminal;

In this case, the view mode supports SubViewIndex, which is index information for identifying at least one screen supported by each view mode for each view mode, and the image capture device supports the subview index when executing the digital PTZ. using the view index.

The view mode is divided into four screens: SourceView displaying the same image as the original image, PanoramaView displaying a 180-degree panoramic image, and Double PanoramaView displaying a 360-degree panoramic image. It is characterized by including a QuadView displaying an image in the form of four quadrants and a Separate View displaying the image in the form of a four-screen division vertical.

As a preferred embodiment of the present invention, the encoding properties are characterized in that they include resolution information, frame rate information, and bit rate information.

As another preferred embodiment of the present invention, a method for providing various view modes in an imaging apparatus includes acquiring a list of view modes supported by a Network Video Transmitter (NVT) in a Network Video Client (NVC); and in the NVC. Checking the view mode of the media profile received from the NVT; and supporting an interface for changing the view mode of the media profile received in the obtained view mode list in the NVC; further comprising, wherein the NVC When the view mode of the received media profile is changed in , encoding properties of the media profile are changed accordingly, and the view mode information is video source configuration information provided by the NVT. It is characterized by being included in.

In the present invention, by providing a method for providing various view modes while complying with the existing ONVIF standard in an image capturing apparatus, there is an advantage in that a client terminal can receive an image in a desired view mode.

FIG. 1 illustrates images received from a client terminal when a fisheye lens camera is installed on a ceiling and FIG. 2 illustrates images received from a client terminal when a fisheye lens camera is installed on a wall.
3 illustrates an example of a multi-sensor camera in which several image sensors are mounted in one camera device.
4 and 5 illustrate an example of a media profile transmitted by an imaging apparatus capable of providing various view modes as a preferred embodiment of the present invention.
6 illustrates an example of providing view mode information in video source environment setting as a preferred embodiment of the present invention.
7 shows an example in which a view mode is applied to a fish-eye lens camera as a preferred embodiment of the present invention.
8 illustrates a process of creating a profile between an NVT 810 and an NVC 820 following the ONVIF standard protocol as a preferred embodiment of the present invention.
9 shows an internal configuration diagram of an image processing apparatus equipped with a dwarping function as another preferred embodiment of the present invention.
FIG. 10 shows an embodiment in which PTZ is performed using a subview index of a view mode in an imaging apparatus as a preferred embodiment of the present invention.
FIG. 11 is a flowchart of a method for providing various view modes in an imaging apparatus as a preferred embodiment of the present invention.
12 is a flowchart of a method for providing various view modes in an imaging apparatus as another preferred embodiment of the present invention.

1 shows an image received from a client terminal when a fisheye lens camera is installed on the ceiling (100), and FIG. 2 shows an image received from a client terminal when a fisheye lens camera is installed on a wall (210, 220), respectively. .

As shown in FIGS. 1 and 2 , in the case of a Fisheye (fisheye lens) camera, when a circular 360-degree image captured by an image sensor is output on a client screen, it is output as an image screen with distortion corrected through a technology called dewarping. did However, in this case, the dewarping method or result is output differently depending on whether the installation location is installed on the ceiling or on the wall.

3 illustrates an example of a multi-sensor camera in which several image sensors are mounted in one camera device. 1 to 3, while the dwarping function is built into the camera of an image capture device such as a fisheye lens camera or a multi-sensor camera, the client displays an image screen captured by a plurality of image sensors transmitted to the client terminal. A standard protocol capable of transmitting in a desired ViewMode form is required.

In a preferred embodiment of the present invention, a protocol for providing a view mode between an image capture device and a client terminal is proposed. In addition, a preferred embodiment of the present invention is characterized in that various view modes provided by the imaging apparatus are compatible with the protocol of ONVIF (Open Network Video Interface Forum), which is used as a security standard protocol. ONVIF is a forum for developing standards for interfaces of IP (Internet Protocol)-based security products, and you can refer to 'http://www.onvif.org/'. The ONVIF protocol may refer to "http://www.onvif.org/onvif/ver10/deviceio.wsdl#op.SetVideoSourceConfiguration".

In addition, an additional ONVIF protocol is required for a method of providing various view modes in the proposed imaging device. It includes GetCompatibleVideoEncederConfiguration contents and SetVideoSourceConfiguration contents, and will be described in detail below.

4 and 5 illustrate an example of a media profile transmitted by an imaging apparatus capable of providing various view modes as a preferred embodiment of the present invention.

Referring to FIG. 4 , a MediaProfile 400 may perform environment settings of various entities (410, 420, 430). Referring to FIG. 5 , a media profile (MediaProfile) 500 may further include various environment setting items such as PTZ environment setting 533 in addition to video environment settings 520 and 530 and audio environment settings 531 and 532. there is.

Referring to FIG. 5 , the media profile 500 includes a video source configuration 520 and a video encoder configuration 530.

The video source configuration (VideoSourceConfiguration) 520 provides image resolution information and lens setting information (Fig. 6, 622 and 623). The Video Source 510 includes a VideoSourceToken representing the ID of the video source 510 and the maximum resolution, frame rate, and usable compression encoding of image data that can be output from each imaging unit or image sensor provided in the image capturing device. including information such as

In a preferred embodiment of the present invention, the view mode 624 may be added in the extension of the video source configuration 620 as shown in the embodiment of FIG. 6 . For details on the view mode (624 in FIG. 6), refer to the description related to FIG. 7. The client terminal can acquire a list of view modes supported by the image processing device and add a command by supporting an interface capable of newly setting a view mode.

In a preferred embodiment of the present invention, a VideoSourceToken is assigned to each image captured by each sensor attached to the image capture device for identification, and a view mode can be set for an image input to one image sensor to which the VideoSourceToken is assigned. there is.

In a preferred embodiment of the present invention, when the view mode (Fig. 6, 624) is changed when setting the video source configuration (VideoSourceConfiguration) (Fig. 6, 620), the video source (VideoSource) (Fig. 5, 510) The input maximum resolution is changed, and encoding property values such as resolution, frame rate, and bit rate of the video encoder configuration (VideoEncoderConfiguration) 530 are changed to correspond to the changed resolution.

In a preferred embodiment of the present invention, in the case of an imaging device equipped with a multi-sensor as in the embodiment of FIG. 2, a plurality of image sensors are attached, a VideoSourceToken is assigned to each image sensor, and an image to which the VideoSourceToken is assigned is attached. A view mode of video source configuration (VideoSourceConfiguration) may be set differently for each image input to the sensor.

7 shows an example in which a view mode is applied to a fish-eye lens camera as a preferred embodiment of the present invention.

As a preferred embodiment of the present invention, a list of view modes supported by the imaging apparatus is as follows. A supportable view mode list may be different for each imaging device. Accordingly, the client terminal receives a view mode list from the image capturing device and a view mode value set for each of at least one image provided by at least one sensor installed in the image capturing device.

The client terminal may newly set a view mode for each received image. When a view mode is newly set in the client terminal, the imaging device changes encoding properties in the media profile to correspond to the newly set view mode. Encoding properties include resolution information, framerate, and bitrate information.

A list of view modes provided in a preferred embodiment of the present invention is as follows.

SourceView (710): A view mode that provides the same image as the original image

PanoramaView (720): A view mode that provides 180-degree panoramic images

DoublePanoramaView (730): A view mode that provides 360-degree panoramic images

QuadView (740): View mode (2*2) that provides images in the form of 4 quadrants divided into 4 screens

SeparateView (750): View mode (1*4) that provides 4 screen divisions as vertical images

In a preferred embodiment of the present invention, each view mode supports SubViewIndex, which is index information for identifying each of at least one screen supported by the view mode (741, 742, 743, 744).

In a preferred embodiment of the present invention, when 'SubViewIndex = 1' is selected as in the embodiment of FIG. PTZ is performed within the given screen. In this case, a fixed value may be used for the subview index (SubViewIndex), or a value may be used as the coordinate system of the output image.

8 illustrates a process of creating a profile between an NVT 810 and an NVC 820 following the ONVIF standard protocol as a preferred embodiment of the present invention.

A network video transmitter (NVT) 810 is a device for transmitting video and audio streams, and includes an imaging device such as an IP camera. A network video client (NVC) 820 is a device that receives and processes video and audio streams, and includes a client terminal. NVC (network video client) may be a personal computer system such as a desktop, laptop, tablet, mini computer, notebook, handheld computer, etc., may be a navigator or digital TV such as IPTV and Smart TV, and a mobile phone, smart phone (smart phone), a terminal for digital broadcasting, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), and a beam projector.

When creating a media profile in ONVIF, the CreateProfileRequest (S810) command is executed in the NVT (810), the CreateProfileResponse (S811) is transmitted from the camera, and the Video Source Configuration is set by executing AddVideoSourceConfigurationRequest (S812).

In a preferred embodiment of the present invention, if GetViewMode (VideoSourceToken) is passed to check the view mode supported by VideoSourceToken before executing AddVideoSourceConfigurationRequest (S812), the camera transmits the view mode supported by VideoSourceToken and the view mode to VideoSourceConfiguration. set And, Video Source Configuration is set in the media profile in the AddVideoSourceConfigurationRequest command (S812).

Also, when changing the view mode in the already set media profile, check the view mode supported by the Video Source Token, change it to the desired view mode, and set the view mode of the desired media profile with SetVideoSourceConfiguration. At this time,

Since the encoding properties of the media profile may vary, the resolution supported by the corresponding media profile, Codec information, and FPS and BPS information can be set through the GetCompatibleVideoEncoderConfigurations command.

* Refer to the protocol for setting SetVideoSourceConfiguration below.

http://www.onvif.org/onvif/ver10/deviceio.wsdl#op.SetVideoSourceConfiguration

* Refer to the protocol for setting GetcompatibleVideoEncoderConfiguration below.

9 shows an internal configuration diagram of an image processing apparatus equipped with a dwarping function as another preferred embodiment of the present invention.

The dewarping unit 910 of the image processing device dewarps the original image received from each image sensor. In this case, the view mode unit 900 performs dwarping according to the preset view mode. In addition, the video processing device performs zoom in the Digital_PTZ unit 920 when there is a request for zoom from the client terminal for the video on which dewarping has been performed. In this case, by using the subview index of the view mode set in the view mode unit 900, as in the embodiment of FIG. can Thereafter, the video is encoded in the encoding unit 930 and streamed to the client terminal. The encoding unit 930 changes encoding properties according to the view mode set in the view mode unit 900 .

11 is a flowchart of a method for providing various view modes in an image capturing apparatus according to a preferred embodiment of the present invention.

The client terminal receives a ViewMode list supported by the imaging device from the imaging device supporting dewarping (S1110). Thereafter, the client terminal receives a media profile conforming to the ONVIF standard from the video recording device and checks the view mode of the received media profile (S1120).

When the view mode of the media profile received from the client terminal is to be modified, an interface capable of newly setting a view mode within the list of view modes supported by the image processing device is supported (S1130). In this case, if the view mode of the media profile received from the client terminal is changed, the encoding property of the streamed media profile is changed accordingly (S1140).

12 is a flowchart of a method for providing various view modes in an imaging apparatus as another preferred embodiment of the present invention.

A list of view modes supported by a Network Video Transmitter (NVT) is obtained from a Network Video Client (NVC) (S1210). Then, the NVC checks the view mode of the media profile received from the NVT (S1220). When changing the view mode in NVC, an interface for changing to a new view mode within the list of view modes supported by NVC is supported (S1230). When the view mode of the media profile received from the NVC is changed, the NVT streams the media profile whose encoding property is changed accordingly (S1240). In this case, the view mode information is characterized in that it is included in video source configuration information provided by the NVT.

The present invention can also be implemented as computer readable codes on a computer readable recording medium. A computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. In addition, the computer-readable recording medium is distributed in computer systems connected through a network, so that computer-readable codes can be stored and executed in a distributed manner.

Optimal embodiments have been disclosed in the above drawings and specifications. Although specific terms have been used herein, they are only used for the purpose of describing the present invention and are not used to limit the scope of the present invention described in the claims. Therefore, those of ordinary skill in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

Claims (6)

Receiving a list of view modes supported by the imaging device from an imaging device supporting dewarping in a client terminal, since a view mode list supportable for each imaging device may be different;
In the client terminal, receiving a media profile conforming to the ONVIF standard from the image capture device and checking a view mode of the received media profile;
Supporting an interface for newly setting the view mode of the received media profile in the received view mode list in the client terminal; and changing the view mode of the received media profile in the client terminal. , the encoding property of the media profile is changed accordingly. The method of claim 1, wherein in the confirming step
The method of providing various view modes in an imaging device, characterized in that the information of the view mode is included in video source configuration information provided by the imaging device. According to claim 1,
When digital PTZ (Pent, Tilt, Zoom) is executed in the client terminal, the video recording device performs dwarping according to the view mode in units of media profiles, and transmits the digital PTZ-executed video to the client terminal. ,
In this case, the view mode supports SubViewIndex, which is index information for identifying at least one screen supported by each view mode for each view mode. using the view index.
The view mode is divided into four screens: SourceView displaying the same image as the original image, PanoramaView displaying a 180-degree panoramic image, and Double PanoramaView displaying a 360-degree panoramic image. Provides various view modes in an imaging device comprising a QuadView displaying an image in a four-quadrant form and a SeparateView displaying an image in a four-screen division vertical form method. The method of claim 1, wherein the encoding property is
A method for providing various view modes in an imaging apparatus, comprising resolution information, frame rate information, and bit rate information. The image capture device of claim 1, wherein the dewarping is supported.
A method for providing various view modes in an imaging apparatus comprising a fisheye camera and a multi-sensor camera equipped with a plurality of image sensors. Acquiring a list of view modes supported by a Network Video Transmitter (NVT) in a Network Video Client (NVC), since the view mode list supportable for each Network Video Transmitter (NVT) may be different; and
Including; checking the view mode of the media profile received from the NVT in the NVC;
Supporting an interface for changing the view mode of the media profile received in the acquired view mode list in the NVC; further comprising,
When the NVC changes the view mode of the received media profile, encoding properties of the media profile are changed accordingly, and the view mode information is video source environment setting information provided by the NVT. A method of providing various view modes in an imaging device, characterized in that included in the configuration.

Images