JP2017216598A - Image processing system, image processing apparatus, and image display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus that can suitably perform image processing when a data stream in which overlap width image data is superimposed on an ancillary area is input thereto.SOLUTION: An image processing apparatus of the present invention comprises: data extension means that creates extended image data by combining image data of an effective image area and overlap width image data included in a data stream; selection means that selects and outputs one image data of the image data of the effective image area and the extended image data; and image processing means that performs image processing on the selected image data.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing system including an imaging device and an image processing device, an image processing device, and an image display device.

  In image processing such as filter processing and edge enhancement processing, the pixel value of the pixel to be processed and the pixel values of the surrounding pixels are used. If there are no peripheral pixels around the pixels at the top, bottom, left, and right edges of the image, end processing that creates a peripheral pixel by copying the pixel values of the end pixels, and the pixel values of multiple pixels near the end Edge processing is known in which peripheral pixels are generated by copying (mirroring) symmetrically.

  Patent Document 1 discloses that image data to which peripheral data (margin) is added is read from a frame memory and image processing is performed.

JP 2008-263646 A

  However, in the above-described conventional technology, it has not been assumed that image processing is performed when a data stream in which margin image data is superimposed as ancillary data on an ancillary area (auxiliary signal area) is input.

  SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide an image processing system, an image processing apparatus, and an image display apparatus that can suitably perform image processing when a data stream in which marginal image data is superimposed on an ancillary area is input. And

  In order to solve the above-described problem, an image processing apparatus according to an embodiment of the present invention is an image processing apparatus in which a data stream in which marginal image data is superimposed on an ancillary area is input, and image data in an effective image area And an input means for inputting a data stream including a marginal image data superimposed on the ancillary area, and the image data of the effective image area and the marginal image data included in the data stream are combined, and extended image data A data expansion unit for generating image data, a selection unit for selecting and outputting one of the image data of the effective image area and the extended image data, and image processing on the image data selected by the selection unit Image processing means to be applied.

  According to the present invention, it is possible to suitably perform image processing when a data stream in which marginal image data is superimposed on an ancillary area is input.

It is a block diagram which shows the structure of the image processing system comprised with an imaging device and an image display apparatus (image processing apparatus). It is a flowchart for demonstrating the process of an imaging device. It is a figure for demonstrating the image data of an effective image area | region, and the marginal image data outside an effective image area | region. It is a figure which shows an example of a structure of the data stream for 1 frame containing the image data of an effective image area | region, and auxiliary data. It is a flowchart for demonstrating the process of an image display apparatus. It is a figure for demonstrating an example of the edge part process by an image process part. It is a figure for demonstrating the other example of the edge part process by an image process part. It is a figure for demonstrating the process which expands a DE (data enable) signal.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The technical scope of the present invention is determined by the claims, and is not limited by the embodiments exemplified below. In addition, not all combinations of features described in the embodiments are essential to the present invention. The contents described in this specification and drawings are illustrative and should not be construed as limiting the present invention. Various modifications (including organic combinations of the embodiments) are possible based on the spirit of the present invention, and they are not excluded from the scope of the present invention. That is, all configurations in which the embodiments and their modifications are combined are also included in the present invention.

(Embodiment 1)
FIG. 1 is a block diagram illustrating a configuration of an image processing system including an imaging device 101 and an image display device (image processing device) 102. The imaging device 101 and the image display device 102 are connected by a transmission cable such as an SDI (Serial Digital Interface) cable. Image data (video data) obtained by photographing the subject with the imaging device 101 is transmitted to the image display device 102 via the SDI cable and displayed on the screen of the image display device 102.

  The imaging device 101 includes an imaging unit 103 and a superimposing unit 104. The imaging unit 103 converts an optical signal obtained by imaging an object with an imaging sensor such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal-Oxide Semiconductor) sensor into an electrical signal.

  The superimposing unit 104 separates the image data output from the imaging unit 103 into image data in the effective image region and marginal image data outside the effective image region. Then, the marginal image data is superimposed on the ancillary area (auxiliary signal area) as auxiliary data, and the image data and auxiliary data of the effective image area are output. The superimposing unit 104 outputs a data stream in which image data and auxiliary data are multiplexed in a format according to the SDI standard defined by SMPTE (Society of Motion Picture and Television Engineers).

  FIG. 2 is a flowchart for explaining the processing of the imaging apparatus 101. In step S201, the optical signal captured by the imaging sensor is converted into an electrical signal and output. Next, in step S202, the superimposing unit 104 superimposes the margin image data on the ancillary area (auxiliary signal area) as auxiliary data. In step S203, the superimposing unit 104 outputs a data stream obtained by multiplexing the image data of the effective image area and the auxiliary data.

  FIG. 3 is a diagram for explaining the image data in the effective image area and the marginal image data outside the effective image area. Around the effective image region (effective pixel region), there are peripheral regions (peripheral pixel regions) L1, L2, and L3 on the left marginal image data, and peripheral regions R1, R2, and R3 on the right marginal image data. Further, there is a peripheral area U1 of the upper marginal image data and a peripheral area U1 of the lower marginal image data.

  FIG. 4 is a diagram illustrating an example of the configuration of a data stream for one frame including image data of an effective image area and auxiliary data. EAV (End of Active Video) indicates the end of the effective image area (the start of the blanking period). SAV (Start of Active Video) indicates the beginning of an effective image area. The marginal image data of the left peripheral regions L1, L2, and L3 and the marginal image data of the right peripheral regions R1, R2, and R3 are superimposed on the ancillary region in the horizontal blanking period. In addition, the marginal image data of the upper peripheral region U1 and the marginal image data of the lower peripheral region U1 are superimposed on the ancillary region in the vertical blanking period. The data stream input to the image display device 102 includes video data composed of continuous frames.

  Referring to FIG. 2 again, the image display apparatus 102 includes an ancillary interpretation unit 105, a data expansion unit 106, a mode setting unit 107, a selection unit 108, an image processing unit 109, a trimming processing unit 110, a selection unit 111, and a display unit 112. Is provided. The image display device 102 may be an image processing device that connects the display unit 112 as a separate body.

  The ancillary interpretation unit 105 interprets auxiliary data superimposed on the ancillary area of the data stream input from the imaging apparatus 101. Then, the image data of the effective image area, the auxiliary data, and the interpretation result are output to the data expansion unit 106 and the selection unit 108, and the interpretation result is output to the selection units 108 and 111.

  Based on the interpretation result of the ancillary interpretation unit 105, the data expansion unit 106 combines the image data of the effective image area and the marginal image data superimposed as auxiliary data, generates expanded image data, and outputs the expanded image data to the selection unit 108 To do.

  The mode setting unit 107 outputs the mode signal set by the user to the selection units 108 and 111. The selection unit 108 selects one of the image data of the effective image area from the ancillary interpretation unit 105 and the extended image data from the data expansion unit 106 based on the setting of the mode setting unit 107 and the interpretation result of the ancillary interpretation unit 105. Select and output the image data.

  The image processing unit 109 performs image processing such as filter processing and edge enhancement processing on the image data output from the selection unit 108 and outputs the result. The image processing unit 109 performs image processing when a DE (data enable) signal is enabled. Then, the image data subjected to the image processing is output to the trimming processing unit 110 and the selection unit 111.

  The trimming processing unit 110 cuts margin image data from the image data output from the image processing unit 109 by trimming processing and outputs the trimmed image data to the selection unit 111.

  The selection unit 111 selects one of the image data output from the trimming processing unit 110 and the image data output from the image processing unit 109 based on the setting of the mode setting unit 107 and the interpretation result of the ancillary interpretation unit 105. Are selected and output to the display unit 112.

  The display unit 112 is configured by, for example, a liquid crystal panel or an organic EL panel. The display unit 112 displays an image on the screen based on the image data output from the selection unit 111.

  The image display device 102 may be a liquid crystal display using a liquid crystal panel or an organic EL display using an organic EL panel. Further, it may be a liquid crystal projector that projects an image on a screen.

  FIG. 5 is a flowchart for explaining the processing of the image display apparatus 102. In step S501, the ancillary interpretation unit 105 interprets auxiliary data superimposed on the ancillary area of the data stream output from the imaging apparatus 101, and determines whether or not the marginal image data is superimposed, the marginal image size, and the like.

  Based on the interpretation result of the ancillary interpretation unit 105, in step S502, it is determined whether the marginal image data is superimposed. If the marginal image data is superimposed, the process proceeds from step S502 to S503. If the marginal image data is not superimposed, the process proceeds from step S502 to S504.

  In step S503, based on the mode signal from the mode setting unit 107, the selection unit 108 selects a memory bandwidth priority mode that prioritizes the frame memory bandwidth and an image processing priority mode that prioritizes image processing without prioritizing the frame memory bandwidth. Determine which is set. If the memory bandwidth priority mode is set, the process proceeds from step S503 to S504. If the image processing priority mode is set, the process proceeds from step S503 to S505. In the memory bandwidth priority mode, it is better that the image size of the image data to be processed is smaller. Therefore, the selection unit 108 selects and outputs the image data of the effective image area output from the ancillary interpretation unit 105. If the image size of the image data to be processed is small, the data transfer amount is small, so that it is possible to suppress the compression of the bandwidth (bandwidth) of the frame memory and to prevent the processing load from increasing. On the other hand, in the image processing priority mode, the selection unit 108 selects and outputs the extended image data output from the data expansion unit 106 in order to perform high-quality image processing.

  In step S504, the image processing unit 109 performs image processing using edge processing on the image data of the effective image area output from the ancillary interpretation unit 105. An example of edge processing will be described with reference to FIGS.

  6 and 7, pixels within the thick frame line are effective pixels in the effective image area (pixels having coordinate values (0, 0) to (x, y) of 0 or more), and are outside the thick frame line. These pixels are peripheral pixels outside the effective image area. FIG. 6 shows an example of end processing for copying (duplicating) pixel values of end pixels to generate a plurality of peripheral pixels. The pixel value of the pixel at the left end is copied to generate a plurality of peripheral pixels in the peripheral region L2 at the left end. Also, the pixel value of the upper left corner pixel is copied to generate a plurality of peripheral pixels in the upper left peripheral region L1, the pixel value of the lower left corner pixel is copied, and the plurality of peripheral pixels in the lower left peripheral region L3 Is generated. Also, the pixel value of the pixel at the right end is copied to generate a plurality of peripheral pixels in the peripheral region R2 at the right end. Also, the pixel value of the pixel in the upper right corner is copied to generate a plurality of peripheral pixels in the peripheral region R1 on the upper right side, the pixel value of the pixel in the lower right corner is copied, and a plurality of peripherals in the peripheral region R3 on the lower right side Generate a pixel. Also, the pixel value of the upper end pixel is copied to generate a plurality of peripheral pixels in the upper end peripheral area U1, the lower end pixel pixel value is copied, and the lower end peripheral area A plurality of peripheral pixels of D1 are generated.

  FIG. 7 shows an example of end processing for copying (mirroring) the pixel values of a plurality of pixels in the vicinity of the end portion symmetrically to generate corresponding peripheral pixels. Specifically, the pixel values of the pixels near the left end are copied to generate corresponding peripheral pixels in the peripheral region L2 at the left end. Further, the pixel value of the pixel near the upper left corner is copied to generate the corresponding peripheral pixel in the upper left peripheral region L1, the pixel value of the pixel near the lower left corner is copied, and the corresponding pixel in the lower left peripheral region L3 Peripheral pixels are generated. Further, the pixel value of the pixel near the right end is copied, and the corresponding peripheral pixel in the peripheral region R2 at the right end is generated. Also, the pixel value of the pixel near the upper right corner is copied to generate the corresponding peripheral pixel in the upper right side peripheral region R1, the pixel value of the pixel near the lower right corner is copied, and the corresponding value in the lower right side peripheral region R3 The peripheral pixels to be generated are generated. Also, the pixel value of the pixel near the upper end is copied, the corresponding peripheral pixel in the peripheral region U1 at the upper end is generated, the pixel value of the pixel near the lower end is copied, and the lower end A peripheral pixel corresponding to the peripheral region D1 is generated.

  In the edge processing as described above, peripheral pixels outside the effective image region are generated by copying, so that the pixel value is different from the actual marginal image data, and the image quality of the image processing result is slightly reduced.

  Referring to FIG. 5 again, in step S505, based on the interpretation result of the ancillary interpretation unit 105, the data expansion unit 106 combines the image data of the effective image area and the marginal image data superimposed as auxiliary data, The extended image data is output to the selection unit 108. The data expansion unit 106 expands the horizontal and vertical DE signals.

  FIG. 8 is a diagram for explaining the process of extending the DE signal. The input vertical DE signal and the input horizontal DE signal are enabled at a timing corresponding to the image data in the effective image area. The extended vertical DE signal and the extended horizontal DE signal are enabled at the timing corresponding to the extended image data obtained by combining the image data in the effective image area and the marginal image data, and the enable period is extended. The data extension unit 106 performs processing for extending the DE signal using information regarding the size of the marginal image obtained from the interpretation result of the ancillary interpretation unit 105.

  Next, in step S506, the image processing unit 109 performs image processing such as filter processing and edge enhancement processing on the extended image data obtained by combining the image data in the effective image region with the marginal image data, and outputs the result. . The image processing unit 109 performs image processing when the extended vertical DE signal and the extended horizontal DE signal are enabled.

  In this case, the image quality of the image processing result is better than when image processing using edge processing is performed in step S504.

  In step S <b> 507, the trimming processing unit 110 cuts margin image data from the image data output from the image processing unit 109 by trimming processing, and outputs the trimmed image data to the selection unit 111. Further, the enable periods of the extended vertical DE signal and the extended horizontal DE signal are reduced, and the trimming vertical DE signal and the trimming horizontal DE signal as shown in FIG. 8 are generated. The enable periods of the extended vertical DE signal and the extended horizontal DE signal are the same as the enable periods of the trimmed vertical DE signal and the trimmed horizontal DE signal.

  In step S <b> 508, the selection unit 111 selects one of the image data output from the trimming processing unit 110 and the image data output from the image processing unit 109 and displays the image data on the display unit 112. Output. Specifically, when the display mode for displaying the effective image area is set by the mode setting unit 107, the image data output from the trimming processing unit 110 is selected and output to the display unit 112. If the mode setting unit 107 sets the display mode to display a peripheral region outside the effective image region, the image data output from the image processing unit 109 is selected and output to the display unit 112.

  As described above, according to the first embodiment, image processing can be suitably performed when a data stream in which marginal image data is superimposed on an ancillary area is input.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or apparatus via a network or a storage medium, and one or more programs in the computer (or CPU, MPU, etc.) of the system or apparatus. This can also be realized by a process in which the processor reads and executes the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions. A computer program itself for realizing one or more functions of the above-described embodiments is also one aspect of the present invention.

DESCRIPTION OF SYMBOLS 101 Image pick-up device 102 Display apparatus 103 Image pick-up part 104 Superimposition part 105 Ancillary interpretation part 106 Data expansion part 107 Mode setting part 108,111 Selection part 109 Image processing part 110 Trimming process part 112 Display part

Claims (5)

An image processing system including an imaging device and an image processing device,
The imaging device
Imaging means for imaging a subject;
Superimposing means for outputting a data stream including the image data of the effective image region and the marginal image data superimposed on the ancillary region based on the image data obtained by the imaging,
The image processing apparatus includes:
Input means for inputting the data stream;
Data expansion means for combining the image data of the effective image area included in the data stream and the marginal image data to generate expanded image data;
Selecting means for selecting and outputting one of the image data of the effective image area and the extended image data;
An image processing apparatus comprising: image processing means for performing image processing on the image data selected by the selection means.   The selection unit selects and outputs one of the image data of the effective image area and the extended image data according to a mode set by a user. The image processing apparatus described. An image processing apparatus to which a data stream in which marginal image data is superimposed on an ancillary area is input,
Input means for inputting a data stream including image data of an effective image area and marginal image data superimposed on the ancillary area;
Data expansion means for combining the image data of the effective image area included in the data stream and the marginal image data to generate expanded image data;
Selecting means for selecting and outputting one of the image data of the effective image area and the extended image data;
An image processing apparatus comprising: image processing means for performing image processing on the image data selected by the selection means.   The selection unit selects and outputs one of the image data of the effective image area and the extended image data according to a mode set by a user. The image processing apparatus described. An image processing apparatus according to claim 3 or 4,
An image display device comprising: display means for displaying an image based on the image data subjected to the image processing.

Images