WO2016151840A1

WO2016151840A1 - Video device capable of capturing video and method for capturing video

Info

Publication number: WO2016151840A1
Application number: PCT/JP2015/059377
Authority: WO
Inventors: 博志砂流
Original assignee: Ｎｅｃディスプレイソリューションズ株式会社
Priority date: 2015-03-26
Filing date: 2015-03-26
Publication date: 2016-09-29
Also published as: JPWO2016151840A1

Abstract

This video device includes: a signal processing unit for generating video data with a resolution for output by scaling input video data; a drive unit for driving an element array unit using the video data with the resolution for output; a first video memory connected to the drive unit; a second video memory connected to the signal processing unit and storing the video data after scaling; and a second signal processing unit for generating a captured image. On receiving an instruction to acquire a captured image, the video device freezes the drive unit and drives the element array unit on the basis of the video data right before freeze, and cancels the scaling process except for the conversion of resolution and transfers the video data stored in the second video memory to the second processing unit for capturing. Then, the video device resets the scaling process except for at least the conversion of resolution and unfreezes the drive unit.

Description

Video device capable of video capture and video capture method

The present invention relates to a video device such as a projector, and more particularly to a video device capable of capturing a projected or displayed video and a method for capturing the video.

Background of the Invention

A video device that projects or displays video such as a projector or a liquid crystal display device operates based on video data output from a video data output device represented by a personal computer, for example. A projector generally includes a liquid crystal panel or digital mirror device having a number of pixels unique to the projector, modulates light for each pixel, and screens the modulated light from the entire liquid crystal panel or the entire digital mirror device. Project to. As a result, an image based on the image data is formed on the screen. The projector includes a display video memory that stores a pixel value for each pixel in a projected video, and a drive unit that drives a liquid crystal panel or a digital mirror device based on the pixel value stored in the display video memory. Are provided.

In recent years, the resolution of video data output from video data output devices has been increasing. For example, in HDMI (High-Definition Multimedia Interface), which is one of the transmission formats of video data, as an example, 1920 pixels in the horizontal direction and 1200 pixels in the vertical direction (this is “1920 × 1200”). High resolution video data is transmitted. When such high-resolution video data is output from the video data output device and the number of pixels of the liquid crystal panel or digital mirror device of the video apparatus is smaller than the resolution, for example, it is described in Patent Document 1. Thus, resolution conversion is performed. Patent Document 2 discloses that in a system in which a plurality of information processing apparatuses and a plurality of image processing apparatuses are connected via a network, jobs are automatically switched between the plurality of apparatuses while performing resolution conversion.

By the way, when a video device such as a projector is used in a meeting or the like, there is a case where it is desired to capture a projected or displayed video as, for example, a screen shot. As a technology for capturing video projected or displayed by a video device, video data input to the video device is converted into image data in JPEG (JointoPhotographic Expart Group) format by software encoding in the video device There is. In order to capture the video displayed or projected by the video device as described above, for example, one frame of video data stored in the display video memory is encoded by software processing, and JPEG Conversion to a format image file has been done. Note that the encoding method used for image file generation is not limited to JPEG.

The video captured by the video device is transferred to another video device via a network, for example. Furthermore, due to the development of the conference system and the spread of the Internet, there are cases where it is used as a conference system between multiple locations by distributing the captured video to the Internet or transmitting the captured video to each other. Yes. Here, there is a case where the resolution of the video device that captures the video is higher than the resolution of the video device that is the transfer destination of the captured video. It is shown that an area having a size corresponding to the resolution of the video apparatus having a lower resolution is extracted and captured with the overlapped area as the center.

When a writable surface such as a whiteboard is used as a screen and the user writes on the screen on which the image is projected using a marker pen, the image including the content written by the user is displayed. Sometimes you want to capture. In Patent Document 4, an image capturing unit that captures a projection surface is provided in a projector, the projection surface is temporarily reduced in brightness, the projection surface is imaged, and the data used for projection and the imaged data are combined and output. Has been shown to do.

In the case of a projector, it is not always possible to place the projector facing the center of the screen. In some cases, the projector is used while being obliquely projected from a projector in at least one of a horizontal direction and a vertical direction with respect to the screen. In the case of such a projection posture, the image on the screen is distorted into a trapezoidal shape, for example, but there are projectors that can automatically compensate and project this trapezoidal distortion (keystone distortion) on the projector side. In Patent Literature 5, image data acquisition processing from an external image source is stopped while the distortion correction amount is determined, and after the correction amount is determined, an image stored in advance in the image storage unit is determined. It is shown that the image is acquired after correction using the corrected amount and the image data acquisition process is restarted at the same time.

JP 2006-106158 A JP 2009-230253 A JP 2009-192670 A International Publication No. 2012/111121 JP 2013-21860 A

As described above, when the resolution of the video data output from the video data output device is as high as 1920 × 1200, for example, the amount of video data to be captured increases, and the captured image is encoded. The subject that the transfer time to an encoder and the processing time of an encoding become long arises. Since the size of the captured image file obtained is also increased, the time required for distributing it is also increased. As a result, when the captured video is transferred to another video device as the conference system, the image distribution interval becomes long. In addition, since the time required for encoding and the transfer time differ depending on the resolution of the video data input to the transmission source video device, there arises a problem that the image update interval in the transfer destination video device is not constant.

For example, when the video apparatus has a trapezoidal distortion correction function, the video stored in the display video memory of the video apparatus is an image after the trapezoidal distortion correction, and the trapezoidal distortion that occurs when projected onto the screen is corrected. The video is distorted in advance so as to cancel. Therefore, when a video stored in the display video memory is captured and used as a captured image, the captured image is also distorted, and there is a problem that it cannot be used in other video devices as it is.

An object of the present invention is to provide a video apparatus capable of shortening the time required to acquire and encode a captured image without being affected by the trapezoidal distortion correction, and a video capturing method in the video apparatus.

According to an exemplary aspect of the present invention, a video device has a number of pixels corresponding to an output resolution, an element array unit that modulates light for each pixel based on video data, and input video data having an input resolution A first signal processing unit that performs scaling processing to generate video data of output resolution, a drive unit that drives the element array unit based on the video data of output resolution, and is connected to the drive unit to connect to the element array unit A first video memory that stores video data for output; a second video memory that is connected to the first signal processing unit and stores at least the scaled video data; and a second video memory A second signal processing unit that generates a captured image based on the video data stored in the video data. When the acquisition of the captured image is instructed, the first signal processing unit Scaling is performed so that the element array unit is driven based on the video data stored in the first video memory immediately before the freeze is executed, and the resolution conversion between the input resolution and the output resolution is excluded. The processing is canceled and the video data stored in the second video memory is transferred to the second processing unit for capture, and then at least scaling processing excluding resolution conversion is reset to freeze the drive unit. To release.

According to another exemplary aspect of the present invention, an image capture method includes an element array unit having a number of pixels corresponding to an output resolution, modulating light for each pixel based on image data, and an input having an input resolution. A signal processing unit that performs video data scaling processing to generate video data of output resolution, a drive unit that drives the element array unit based on the video data of output resolution, and is connected to the drive unit to the element array unit In a video apparatus comprising: a first video memory that stores video data for output; and a second video memory that is connected to a first signal processing unit and stores at least video data that has undergone scaling processing In the video capturing method, when acquisition of a captured image is instructed, the drive unit is frozen and stored in the first video memory immediately before the freeze is executed. Based on the video data stored in the second video memory by releasing the scaling processing except resolution conversion between the input resolution and the output resolution. Performing capture, and after acquiring the captured image, resetting the scaling process excluding at least resolution conversion to release the freeze of the drive unit.

According to the present invention, when, for example, horizontal / vertical trapezoidal distortion correction is performed on an image to be projected or displayed, the video is frozen at a later stage than the signal processing unit that performs the trapezoidal distortion correction, and during the freeze period. By canceling the trapezoidal distortion correction, and then resetting the keystone distortion correction, the image that is actually projected or displayed is corrected to the trapezoidal distortion correction, and the distortion is not affected by the keystone distortion correction. No capture image can be obtained. In addition, even when the input resolution is high, a capture image having the same resolution as the output resolution of the video device is generated. Therefore, the video data transfer time for capture, the time required for encoding the capture image, and the time required for delivery of the capture image are reduced. Can be shortened. Regardless of the input resolution, captured images can be distributed at regular time intervals.

It is a block diagram which shows the video device of one Embodiment of this invention. It is a figure which shows the screen by input video data. It is a figure which shows the screen by the video data after resolution conversion, and trapezoid distortion correction. 2 is a flowchart for explaining a capture operation in the video apparatus shown in FIG. 1.

Next, embodiments of the present invention will be described with reference to the drawings. FIG. 1 shows a video apparatus according to an embodiment.

The video apparatus shown in FIG. 1 projects a video on a screen or displays it on a screen based on video data output from a video data output device such as a personal computer. Specifically, it is a projector or the like. Here, as an example, it is assumed that the video data input to this video device is in the HDMI format having a resolution of 1920 pixels × 1200 pixels, and the resolution of the video projected on the screen by this video device is 1280 pixels × 800. Assume that it is a pixel. Therefore, in the video device, the resolution of the video is converted. Further, in this video apparatus, it is assumed that horizontal / vertical (H / V) trapezoidal distortion correction is also performed. Note that both resolution conversion and trapezoidal distortion conversion are processing for mapping each pixel of input video data to each pixel in a video actually projected or displayed by the video device by some arithmetic processing. Therefore, resolution conversion and trapezoidal distortion correction are collectively referred to as scaling. As will be described later, scaling may include things other than resolution conversion and trapezoidal distortion correction.

The video apparatus includes video data in HDMI format, that is, an HDMI decoding unit 101 that receives and decodes an HDMI signal, and a remote control receiving unit 102 that receives a signal from a remote controller (remote controller) for operating the video apparatus. The signal processing unit 103 that performs processing such as scaling on the video data decoded in the HDMI decoding unit 101, the signal processing video memory 104 connected to the signal processing unit 103, and the resolution of the video device An element array unit 105 that includes a liquid crystal panel or a digital mirror device with the number of pixels to be modulated and modulates light from the light source for each pixel, a drive unit 106 that drives each pixel of the element array unit 105, and a drive unit 106 Display video memory 107. The remote control receiving unit 102 transmits the received remote control signal to the signal processing unit 103. The driving unit 106 drives the element array unit 106 on the basis of the video data scaled by the signal processing unit 103 so that projection or display is performed based on the scaled video data, and temporarily. The signal processing unit 103 has a function of stopping input of video data, that is, a function of freezing. The display video memory 107 is used to temporarily store the video data input to the drive unit 106 and convert it into a format for driving the element array unit 106. The resolution in the input HDMI format video data is called input resolution, and the resolution of the video apparatus itself, that is, the resolution determined by the element array unit 105 is called output resolution.

In this video apparatus, when a user performs some writing operation on the projected screen, a captured image including contents written by the user can be acquired. Assuming that the user performs a writing operation on the projected image using the electronic pen 120, the video apparatus includes an electronic pen detection unit 108 that detects characters written with the electronic pen 120 and an electronic pen 120. There is further provided a multimedia (MM) processing unit 110 that performs processing such as video composition so that written characters are projected onto the input video data. The signal processing unit 103 corresponds to the first signal processing unit, while the multimedia processing unit 110 corresponds to the second signal processing unit. Accordingly, the display video memory 107 corresponds to a first memory, and the signal processing video memory 104 corresponds to a second memory.

The multimedia processing unit 110 further has a function of capturing a video to be projected as a captured image and distributing it to other video devices and the like via a network. The multimedia processing unit 110 includes a video composition unit 111, a JPEG encoder unit 112, and a video distribution unit 113. The video composition unit 111 synthesizes the characters detected by the electronic pen detection unit 108 and the scaled video to be output to the element array unit 105. The synthesized image is the signal processing unit 103. And the JPEG encoding unit 112. The JPEG encoding unit 112 encodes the video synthesized by the video synthesizing unit 111 into a JPEG image file by software processing. The video distribution unit 113 distributes the encoded JPEG image file via a network such as the Internet.

Here, the signal processing unit 103 and the signal processing video memory 104 will be described in more detail. The signal processing unit 103 includes a CPU (central processing unit), and the resolution (input resolution) of the video data input from the HDMI decoding unit 101 is set to the resolution (output resolution) of the element array unit 105. In addition to performing resolution conversion processing for conversion to (), the video data from the HDMI decoding unit 101 and the video data from the M video synthesis unit 111 of the multimedia processing unit 110 are selected according to the remote control signal from the remote control reception unit 102. The selected video data is to be output to the drive unit 106. Further, the signal processing unit 103 performs correction by horizontal / vertical trapezoidal distortion correction when the video device as a projector is installed obliquely with respect to the screen. The signal processing video memory 104 is used for storing data in the processing in the signal processing unit 103, and is a remote control signal among the video data input from the HDMI decoding unit 101 and the video data from the video synthesis unit 111. And the video data after scaling the selected video data is stored.

2 and 3 show screens represented by video data handled by the video apparatus of the present embodiment. FIG. 2 shows a screen 151 having a resolution of 1920 pixels × 1200 pixels input as an HDMI signal. In FIG. 3, a dotted line indicates a screen 152 of 1280 pixels × 800 pixels that is the resolution of the element array unit 105, and a solid line indicates the signal processing video memory 104 in a state where horizontal / vertical trapezoidal distortion correction is performed. A screen 153 based on stored video data, that is, a video area actually output to the element array unit 105 for projection is shown.

When horizontal / vertical trapezoidal distortion correction is performed, the data that is scaled for output and stored in the signal processing video memory 104 is shown by a solid line in FIG. 3 for horizontal / vertical trapezoidal distortion correction. The video data has a contour distorted from the original rectangle. When this video data is captured as it is, the captured image is also distorted. Therefore, in this embodiment, when capturing, the drive unit 106 freezes the screen using the video stored in the display video memory 107, and then the video data stored in the signal processing video memory 104. The horizontal / vertical trapezoidal distortion correction is canceled, and the video data in the signal processing video memory 104 is transferred to the multimedia processing unit 116 to capture the video. When the transfer of the video data to the multimedia processing unit 116 is completed, the horizontal / vertical trapezoidal distortion correction is reset and the freeze in the driving unit 106 is also released. The capture instruction by the user is given by a remote control signal received by the remote control receiving unit 102. The signal processing unit 103 controls execution and release of freeze in the drive unit, release and reset of horizontal / vertical trapezoidal distortion correction, and data transfer to the multimedia processing unit 116 based on a remote control signal indicating a capture instruction.

As described above, in the present embodiment, when capturing of video data is instructed, the drive unit 106 is frozen, and as a result, the element array unit 105 is driven with the contents of the display video memory 107 immediately before the freeze, The content immediately before the freeze will continue to be projected or displayed. On the other hand, in the signal processing video memory 104, video data whose resolution is converted to that of this video apparatus but not subjected to horizontal / vertical trapezoidal distortion correction, that is, video data having the same aspect ratio as the input video data (see FIG. 3, the video data corresponding to the screen 152 indicated by a dotted line is stored. Therefore, if capture processing is performed based on the video data stored in the signal processing video memory 104 at this time, a capture image that is not affected by the trapezoidal distortion correction and that is captured by the resolution of the video device can be obtained. It will be. Since it is a captured image with the resolution of this video device, the time required to acquire video data for JPEG encoding, the time required for encoding itself, and the time required for distributing the captured image file can be reduced. . Furthermore, in the present embodiment, since two processing units, the signal processing unit 103 and the multimedia processing unit 110, are used, video data is transferred from the signal processing unit 103 to the multimedia processing unit 110 even when capturing is performed. When the image processing is completed, the video data sent to the element array unit 105 is released by releasing the freeze in the driving unit 106 regardless of the progress of the encoding process in the multimedia processing unit 110. Can do.

FIG. 4 illustrates processing in the signal processing unit 103 of the video apparatus according to the present embodiment. Here, it is assumed that the signal processing unit 103 selects video data having a resolution of 1920 × 1200 from the HDMI decoding unit 101 for output by remote control signal input. If there is a capture execution request by remote control signal input, the signal processing unit 103 outputs a freeze execution command to the drive unit 106 in step 201. In response to this freeze execution command, the drive unit 106 interrupts the input of the video data from the signal processing unit 103, and the video data stored in the display video memory 107 storing the video immediately before the freeze execution is stored in the element array unit 105. Continue to output. Since the input resolution is 1920 × 1200 and the resolution (output resolution) of the element array unit 105 is 1200 × 800, the signal processing unit 103 performs resolution conversion processing. There may be a trapezoidal distortion correction. When the horizontal / vertical trapezoidal distortion correction is performed, the video data in the signal processing video memory 104 is distorted as shown by a solid line in FIG.

Therefore, the signal processing unit 103 determines whether or not scaling processing other than the resolution conversion processing is performed in Step 202. If not, the process proceeds to Step 204. If it is performed, Step 203 is performed. In FIG. 8, the currently set horizontal / vertical trapezoidal distortion correction values are stored, the horizontal / vertical trapezoidal distortion correction is canceled, and then the process proceeds to step 204. As a result, in the signal processing video memory 104, video data having a resolution of 1280 × 800 and having the horizontal / vertical trapezoidal distortion correction canceled is stored.

In step 204, the signal processing unit 103 acquires video data that has not undergone trapezoidal distortion correction in the signal processing video memory 104, and in step 205, the acquired video data is converted into the video synthesis unit 111 of the multimedia processing unit 110. Forward to. As a result, the multimedia processing unit 110 starts capturing video data and JPEG encoding.

When the transfer of the video data to the video composition unit 111 is completed, the signal processing unit 103 determines in step 206 whether scaling processing other than the resolution conversion processing is performed. If YES in step 207, horizontal / vertical trapezoidal distortion correction is reset in step 207 based on the previously stored correction value, and the flow advances to step 208. In step 208, the signal processing unit 103 outputs a freeze release command to the driving unit 106, whereby the driving unit 106 resumes input of video data from the signal processing unit 103.

As described above, in the present embodiment, when horizontal / vertical trapezoidal distortion correction is performed, the image is projected or displayed using the image immediately before the freeze by executing the freeze in the driving unit 106. / The user can see the video on which the vertical trapezoidal distortion correction has been performed. For example, in the case of a conference using a projector, the conference participants can see the screen projected on the screen without a sense of incongruity. On the other hand, the video composition unit 111 of the multimedia processing unit 110 that generates the capture image is input with the video that has not been subjected to horizontal / vertical trapezoidal distortion correction. An image without distortion is not affected by the trapezoidal distortion correction. Further, since the video composition unit 111 receives the video data of the resolution (output resolution) from the element array unit 105, the processing can be executed with a smaller data amount than the input resolution. Therefore, the transfer time of video data to the video composition unit 111, the time required for JPEG encoding, and the delivery time of the captured image can be shortened, and delivery can be performed at regular time intervals regardless of the input resolution.

In the above description, horizontal / vertical trapezoidal distortion correction is cited as scaling processing other than resolution conversion, but scaling processing other than resolution conversion is not limited to this. The pincushion that corrects depending on the installation state of the projector, screen enlargement, screen reduction, and 4-point correction are also included in the scaling processing here. Here, the four-point correction is a process of deforming the output video by designating three points other than the reference point (for example, the lower right point) among the four corners (upper left, lower left, upper right, lower right) of the output video. is there.

DESCRIPTION OF SYMBOLS 101 HDMI decoding part 102 Remote control receiving part 103 Signal processing part 104 Signal processing video memory 105 Element array part 106 Drive part 107 Display video memory 108 Electronic pen detection part 110 Multimedia (MM) processing part 111 Video composition part 112 JPEG encoding Part 113 Video distribution part 120 Electronic pen

Claims

An element array unit having a number of pixels corresponding to the output resolution and modulating light for each pixel based on video data;
A first signal processing unit that performs scaling processing on input video data having an input resolution to generate video data of the output resolution;
A drive unit for driving the element array unit based on the video data of the output resolution;
A first video memory connected to the drive unit and storing video data for output to the element array unit;
A second video memory connected to the first signal processing unit and storing at least the video data subjected to the scaling process;
A second signal processing unit that generates a captured image based on video data stored in the second video memory;
Have
When the capture of the captured image is instructed, the first signal processing unit executes the freeze of the drive unit and based on the video data stored in the first video memory immediately before the execution of the freeze The element array unit is driven, and the scaling process except for the resolution conversion between the input resolution and the output resolution is canceled, and the second video memory stores the video data stored in the second video memory for capture. A video apparatus that transfers to a processing unit, and then resets at least scaling processing excluding the resolution conversion to release the freeze of the driving unit.
The scaling process excluding the resolution conversion includes a trapezoidal distortion correction process,
The correction amount of the trapezoidal distortion correction is stored when the scaling process excluding the resolution conversion is canceled, and the stored correction amount is used when the scaling process excluding the resolution conversion is reset. The video device described in 1.
A detection unit for detecting a user's writing on the video generated by the element array unit;
The second signal processing unit includes a video synthesis unit that superimposes the detection result of the detection unit on the scaled video data.
The video device according to claim 1, wherein the element array unit is driven based on video data output from a video synthesis unit, and the captured image is generated.
The video device according to claim 3, wherein the detection unit is an electronic pen detection unit that detects an electronic pen of the user.
5. The video apparatus according to claim 1, wherein the second signal processing unit includes a video distribution unit that distributes the captured image.
An element array unit that has the number of pixels corresponding to the output resolution and modulates light for each pixel based on the video data, and performs scaling processing on the input video data having the input resolution to generate video data of the output resolution A signal processing unit; a driving unit that drives the element array unit based on video data of the output resolution; and a first unit that is connected to the driving unit and stores video data for output to the element array unit A video capture method in a video device, comprising: a video memory; and a second video memory connected to the first signal processing unit and storing at least the video data subjected to the scaling process,
When acquisition of the captured image is instructed, the element array unit is driven based on the video data stored in the first video memory immediately before the freeze is executed by executing the freeze of the drive unit. And canceling the scaling process excluding the resolution conversion between the input resolution and the output resolution and executing capture based on the video data stored in the second video memory;
After acquiring the captured image, reset the scaling process excluding at least the resolution conversion to cancel the freeze of the drive unit;
Having a method.
The scaling process excluding the resolution conversion includes a trapezoidal distortion correction process,
7. The correction amount of the trapezoidal distortion correction is stored when the scaling process excluding the resolution conversion is canceled, and the stored correction amount is used when the scaling process excluding the resolution conversion is reset. The method described in 1.
Detecting user writing on the video generated by the element array unit;
Superimposing the detected writing on the scaled video data;
Further comprising
The method according to claim 6 or 7, wherein the element array unit is driven based on the video data on which the writing is superimposed, and the captured image is generated.
The method according to any one of claims 6 to 8, further comprising delivering the captured image.
The method according to any one of claims 6 to 9, comprising encoding the captured image based on a JPEG format.