JP2006303623A - Image processing controller, electronic apparatus, and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing controller or the like capable of carrying out overlapping processing in real time by suppressing an increase in an information amount to be stored even when an image size is increased. <P>SOLUTION: The image processing controller 100 includes: an image data input interface to which image data of an input image are inputted; an overlapping processing section 130 for carrying out overlapping processing between frame data and the image data of the input image for the purpose of displaying a background image; a display memory 140 to which image data after the overlapping processing carried out by the overlapping processing section 130 are written; and an image data output interface for outputting image data read from the display memory 140. The frame data are image data of a frame image designated with a region wherein the input image is overlapped on the background image. The overlapping processing section 130 carries out the overlapping processing for overlapping the input image into the region. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing controller, an electronic device, and an image processing method.

  2. Description of the Related Art In recent years, mobile phones as portable electronic devices display images on a display unit based on image data included in communication data transmitted and received, and display images captured by a built-in imaging unit on a display unit. It can be done. One such image display on the display unit is a superimposed display.

  In the superimposed display, for example, an image of the image data or a captured image included in the communication data can be displayed in a preset background image. This can be realized by performing a superimposition process between the image data of the background image and the image data included in the communication data, and displaying on the display unit based on the processed image data.

The overlay process is disclosed in Patent Document 1, for example. In Patent Document 1, bit information indicating whether or not overlay display is performed is stored in a memory in correspondence with a display pixel position, and bit information is sequentially read out in synchronization with display scanning, and two image data are superimposed. Techniques to do are disclosed.
Japanese Patent Laid-Open No. 5-289651

  However, if bit information is provided corresponding to the display pixel position, bit information for one screen is required in addition to image data for one screen. Therefore, there is a problem that as the image size increases, the capacity of bit information to be stored for performing the overlay process increases.

  In addition, when superimposing processing is performed on image data of a captured image, it is necessary to display on the display unit in real time based on the processed image data. For this reason, the overlay process must be performed in real time.

  Further, even when compression processing is performed and transmission is performed as communication data, it is desirable to perform compression processing on the same image data as the image data of the display image. The difference between the image viewed on the display unit and the image actually subjected to the compression process is that the usability is deteriorated for the user. Therefore, it is desirable that the image data after the superimposition processing is the same as the image data to be displayed on the display unit.

  As described above, real-time processing is required for the image data after superimposition processing, but real-time processing cannot be performed when image data to be superposed is stored in a memory.

  The present invention has been made in view of the technical problems as described above. The object of the present invention is to perform an overlay process while suppressing an increase in the amount of information to be stored even when the image size increases. An image processing controller, an electronic apparatus, and an image processing method that can be performed are provided.

  Another object of the present invention is to provide an image processing controller, an electronic device, and an image processing method for performing superimposition processing in real time while suppressing an increase in the amount of information to be stored even when the image size increases. is there.

In order to solve the above problems, the present invention
An image processing controller for superimposing two images,
An image data input interface through which image data of the input image is input;
An overlay processing unit that performs overlay processing of frame data and image data of the input image to display a background image;
A display memory in which image data after the overlay processing performed by the overlay processing unit is written;
An image data output interface for outputting image data read from the display memory,
The frame data is
The image data of a frame image in which an area for superimposing the input image in the background image is designated,
The superposition processing unit relates to an image processing controller that performs superposition processing for superimposing the input image on the area.

  According to the present invention, it is only necessary to superimpose the input image on the area designated in the frame image, so that an increase in the amount of information to be stored can be suppressed even when the image size increases. It is possible to provide an image processing controller that realizes superimposing processing that can be performed.

  In addition, since the superimposition processing unit for performing the above superimposition processing is provided in the front stage of the display memory, the superimposition processing can be executed in real time unlike the case where the image data is temporarily stored in the memory.

The present invention also provides
An image processing controller for superimposing two images,
A camera interface to which image data of a captured image is input;
A host interface for inputting image data from the host;
An overlay processing unit that performs overlay processing of frame data for displaying a background image and image data of an input image input via the camera interface or the host interface;
A display memory in which image data after the overlay processing performed by the overlay processing unit is written;
An image data output interface for outputting image data periodically read from the display memory,
The frame data is
The image data of a frame image in which an area for superimposing the input image in the background image is designated,
The superposition processing unit relates to an image processing controller that performs superposition processing for superimposing the input image on the area.

  According to the present invention, it is only necessary to superimpose the input image on the area designated in the frame image, so that an increase in the amount of information to be stored can be suppressed even when the image size increases. It is possible to provide an image processing controller that realizes superimposing processing that can be performed.

  In addition, since the superimposition processing unit for performing the above superimposition processing is provided in the front stage of the display memory, the superimposition processing can be executed in real time unlike the case where the image data is temporarily stored in the memory.

In the image processing controller according to the present invention,
Including an image compression / decompression unit that performs decompression processing of the image data subjected to compression processing,
The overlay processing unit
The image data after the decompression process performed by the image compression / decompression unit and the frame data are superimposed.
The image data after the overlay process may be written in the display memory.

  Further, according to the present invention, it is possible to execute the superimposition process in real time even for the image data subjected to the compression process.

In the image processing controller according to the present invention,
A source selection register for designating the image data of the captured image or the image data after the decompression process as source data to be superimposed with the frame data;
And a path selection register for designating whether the image data after the superimposition processing is written in the display memory or supplied to the compression processing by the image compression / decompression unit.

  According to the present invention, it is possible to write image data after superimposition processing performed in real time into a display memory for display processing or for compression processing.

In the image processing controller according to the present invention,
Based on the setting data of the path selection register, the image data after the overlay process is written to the display memory, and the image data is supplied to the compression process by the image compression / decompression unit,
The image data written in the display memory and the image data compressed by the image compression / decompression unit may be the same.

  According to the present invention, in addition to the above effects, the display image and the image of the image data to be compressed can be made the same, so that it is possible to provide an image processing controller that is convenient for the user.

In the image processing controller according to the present invention,
A frame data storage unit for storing the frame data;
Including an interrupt generation unit for generating an interrupt signal,
The overlay processing unit
A read buffer for buffering the frame data read from the frame data storage unit;
The interrupt generation unit can generate the interrupt signal on the condition that an instruction to read frame data from the read buffer is issued even though the read buffer is empty.

  According to the present invention, it is possible to notify, as an interrupt signal, a situation in which execution of the overlay process is started in a state where the overlay processing unit is not supplied with frame data or image data of an input image. Therefore, it is possible to prevent the execution of useless superimposition processing.

In the image processing controller according to the present invention,
The frame data is
Image data of a frame image in which a key color is set in a region where the input image or the image data of the decompressed image data is superimposed;
The overlay processing unit
Of the frame data, the image data of the pixels in the area where the key color is set can be replaced with the image data of the input image or the image data after the decompression process.

  According to the present invention, since it is not necessary to have bit information for one screen in addition to image data for one screen in order to perform the superimposition processing, the superimposition processing is performed even if the image size increases. It is possible to suppress an increase in the capacity of bit information to be stored for execution.

The present invention also provides
A display device;
Any one of the image processing controllers described above;
The present invention relates to an electronic apparatus including a display driver that drives the display device based on image data supplied by the image processing controller.

  According to the present invention, it is possible to provide an electronic apparatus including an image processing controller that performs an overlay process in real time while suppressing an increase in the amount of information to be stored even when the image size increases.

The present invention also provides
An image processing method for superimposing two images,
Set the frame data for displaying the background image in the frame data storage unit,
While reading the frame data from the frame data storage unit in synchronization with the image data of the captured image, the image data of the captured image and the frame data are overlaid,
Write the image data after the overlay process to a display memory,
The image data read from the display memory is periodically read,
The frame data is
The present invention relates to an image processing method which is image data of a frame image in which an area where an input image is superimposed on the background image is designated.

The present invention also provides
An image processing method for superimposing two images,
Set the frame data for displaying the background image in the frame data storage unit,
The decompressed image data and the frame data are superimposed while reading out the frame data from the frame data storage unit in synchronization with the decompressed image data obtained by decompressing the compressed image data. Process
Write the image data after the overlay process to a display memory,
The image data read from the display memory is periodically read,
The frame data is
The present invention relates to an image processing method which is image data of a frame image in which an area where an input image is superimposed on the background image is designated.

In the image processing method according to the present invention,
The image data after the overlay process is written to the display memory, and the image data is compressed.
The image data written to the display memory and the image data to be compressed may be the same.

In the image processing method according to the present invention,
An interrupt notification can be made on condition that an instruction to read frame data from the read buffer is issued even though a read buffer for buffering the frame data read from the frame data storage unit is empty. .

In the image processing method according to the present invention,
The frame data is image data of a frame image in which a key color is set in a region where the input image or the image data of the decompressed image data is superimposed,
Of the frame data, the image data of the pixels in the area where the key color is set can be replaced with the image data of the input image or the image data after the decompression process.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The embodiments described below do not unduly limit the contents of the present invention described in the claims. Also, not all of the configurations described below are essential constituent requirements of the present invention.

1. Image Processing Controller FIG. 1 is a block diagram showing a configuration example of an image processing controller in the present embodiment. Note that the image processing controller 100 according to the present embodiment does not include all the blocks illustrated in FIG. 1 and may have a configuration in which some of the blocks illustrated in FIG. 1 are omitted.

  The image processing controller 100 functions as a display controller that supplies image data to a display driver that drives a display device typified by an electro-optical device such as an LCD (Liquid Crystal Display) panel. Here, the image processing controller 100 can superimpose two images and supply the image data after the superimposition processing to the display driver.

  Therefore, the image processing controller 100 includes a camera I / F 110 and a host I / F 120 that function as an image data input interface (InterFace: hereinafter referred to as I / F), an overlay processing unit 130, a display memory 140, and image data. And an LCD I / F 150 functioning as an output I / F. Here, the image processing controller 100 may have a configuration in which one of the camera I / F 110 and the host I / F 120 is omitted.

  Image data of the input image is input to the image data input I / F. When the image data input I / F is a camera I / F 110, image data of an image captured by a camera module as an imaging unit (image data of an input image in a broad sense) is input to the camera I / F 110. That is, image data in the effective pixel area of the image sensor of the camera module is input to the camera I / F 110. More specifically, the camera I / F 110 performs interface processing of the image data (reception processing with the camera module and signal buffering), and the image data after the interface processing is sent to the overlay processing unit 130. Can be supplied.

  When the image data input I / F is the host I / F 120, image data from the host (image data of the input image in a broad sense) is input to the host I / F 120. That is, the host I / F 120 includes image data generated by a host (not shown) including a central processing unit (hereinafter abbreviated as CPU) and a memory, and an image processing controller 100 from the host. Control data for controlling is input. The CPU performs predetermined processing such as control of the image processing controller 100 and display control of the display device according to a program stored in the memory. At this time, the host I / F 120 can perform interface processing (reception processing with the host and signal buffering), and supply image data after the interface processing to the superimposition processing unit 130. The host I / F 120 receives image data read from the display memory 140 and the like and an interrupt signal. The host I / F 120 can perform interface processing (transmission processing with the host and signal buffering) and output image data and interrupt signals after the interface processing to the host.

  The overlay processing unit 130 performs overlay processing of the frame data and the image data of the input image. This frame data is data for displaying a background image, and is image data of a frame image in which an area for superimposing an input image is specified in the background image. Then, the superimposition processing unit 130 performs superimposition processing for superimposing the input image on the region.

  FIG. 2 is a schematic explanatory diagram of the overlay process of the overlay processor 130 of FIG.

  The frame data FD is image data of an image including, for example, a person image PD1 as a background image. In this image, an area AR for superimposing the input image is designated. A key color is set for each pixel in the area AR. On the other hand, the input image data ID is image data of an image including the person image PD2 as an image.

  The size of the frame image represented by the frame data FD and the size of the input image represented by the input image data ID match, and the position of each pixel of the frame image corresponds to the position of each pixel of the input image. ing. In FIG. 2, the area AR in the frame image of the frame data FD corresponds to the area in which the person image PD2 in the input image of the input image data ID is arranged.

  The superimposition processing unit 130 performs processing for superimposing two images based on the frame data FD and the input image data ID. More specifically, the superimposition processing unit 130 realizes a superimposition process of two images by replacing the area AR of the frame image of the frame data FD with the person image PD2 of the input image of the input image data ID. . Therefore, the image represented by the image data OD after the overlay processing includes person images PD1 and PD2, and the person image PD2 is included in the area AR1 in the image represented by the image data OD.

  In FIG. 1, the display memory 140 is written with the image data after the overlay processing performed by the overlay processing unit 130 in this way. The LCD I / F 150 periodically reads image data from the display memory 140 and outputs the image data.

  The LCD I / F 150 outputs image data periodically read from the display memory 140 to a display driver (not shown) that drives the display device. The LCD I / F 150 performs image data interface processing (transmission processing with the display driver and signal buffering), and outputs the image data after the interface processing to the display driver.

  Further, the image processing controller 100 can include a JPEG (Joint Photographic Coding Experts Group) circuit 160 as an image compression / decompression processing unit. The JPEG circuit 160 can perform compression processing and decompression processing of image data according to the JPEG standard. When image data subjected to compression processing is input via the host I / F 120, the JPEG circuit 160 performs decompression processing on the image data subjected to the compression processing. Then, the superimposition processing unit 130 performs superimposition processing on the decompressed image data and frame data.

  Also, the JPEG circuit 160 performs compression processing on the image data after the overlay processing performed by the overlay processing unit 130, and causes the host or the like to output the image data after the compression processing via the host I / F 120. Can do.

  Further, the image processing controller 100 can include an RGB-YUV conversion circuit (hereinafter abbreviated as RYC) 170, a resizer 180, and a YUV-RGB conversion circuit (hereinafter abbreviated as YRC) 190.

  The image data input to the camera I / F 110 is in YUV format. In contrast, the image data input to the host I / F 120 is in RGB format. Accordingly, the RYC 170 can convert RGB format image data input to the host I / F 120 into YUV format image data.

  The resizer 180 also enlarges or reduces the image size of the image represented by the image data, and generates image data of the image after the enlargement or reduction. When enlarging the image size, the resizer 180 performs a process of enlarging the image size while adding interpolation pixels between pixels arranged in at least one of the horizontal direction and the vertical direction of the original image represented by the image data, for example. When reducing the image size, for example, the resizer 180 thins out pixels arranged in at least one of the horizontal direction and the vertical direction of the original image represented by the image data, and performs image processing while performing interpolation processing on the thinned pixels. Process to reduce.

  Such a resizer 180 can perform processing for enlarging or reducing the size of the image represented by the image data on the image data input via the camera I / F 110. The resizer 180 can perform processing for enlarging or reducing the size of the image represented by the image data, on the image data converted into the YUV format by the RYC 170. The output of the resizer 180 is supplied to the overlay processing unit 130. Therefore, the overlay processing unit 130 can perform overlay processing on two pieces of image data in the YUV format.

  The YRC 190 converts the image data in the YUV format that has been subjected to the overlay processing by the overlay processing unit 130 into image data in the RGB format. The RGB format image data converted by the YRC 190 is written into the display memory 140.

  The display memory 140 has a plurality of storage areas. More specifically, the storage area of the display memory 140 includes a frame data storage area FAR, an image data storage area IAR, a line buffer area LBR, and an image data storage area JAR for compression / decompression processing.

  The frame data storage area FAR of the display memory 140 functions as a frame data storage unit, and frame data is stored in the frame data storage area FAR. This frame data is set by the host via the host I / F 120, for example.

  Image data for driving the display device is stored in the image data storage area IAR. This image data is the image data after the superposition process converted into the RGB format by YRC190. The LCD I / F 150 reads out image data from the image data storage area IAR in accordance with the refresh cycle of the display device, and outputs it to a display driver that drives the display device.

  The line buffer area LBR and the compression / decompression image data storage area JAR are work areas for image data compression processing or decompression processing by the JPEG circuit 160. For example, the host sets image data that has undergone compression processing via the host I / F 120 in the compression / decompression processing image data storage area JAR, and the JPEG circuit 160 stores image data from the compression / decompression processing image data storage area JAR. Are read out, decompression processing is performed, and the processing result is written in the line buffer area LBR. Then, the resizer 180 reads the image data after the decompression process from the line buffer area LBR. Further, for example, after the image data after the overlay processing performed by the overlay processing unit 130 is written in the line buffer area LBR, the JPEG circuit 160 reads the image data from the line buffer area LBR and performs compression processing. The processing result is written into the compression / decompression image data storage area JAR. Then, the host reads out the compressed image data from the compression / decompression processing image data storage area JAR via the host I / F 120.

  Further, the image processing controller 100 can include an interrupt generation unit 200. For example, when the interrupt generation unit 200 is instructed to execute the superimposition process in a state where the frame data read from the frame data storage area FAR or the image data of the input image is not supplied to the superimposition processing unit 130, An interrupt signal for issuing an interrupt notification is generated. This interrupt signal is transmitted to the host via the host I / F 120.

  For example, when each area of the display memory 140 is accessed by sharing one bus, the access may conflict. In this case, due to a delay due to competition, the overlay processing unit 130 may start executing the overlay process in a state where no frame data or image data of an input image is supplied. In such a situation, the overlay process cannot be executed, so that the host can be notified by an interrupt. Therefore, the host can avoid such a situation and prevent execution of useless superimposition processing.

  Such an image processing controller 100 is controlled based on the control data of each control register of the control register unit 210. Each control register of the control register unit 210 can be accessed by the host via the host I / F 120.

  FIG. 3 shows an outline of the configuration of the control register unit 210 of FIG.

  The control register unit 210 includes a key color setting register 212, a source selection register 214, a path selection register 216, and an activation register 218.

  In the key color setting register 212, control data corresponding to the key color of the area AR of the frame data FD in FIG. A key color designation signal Kc is output corresponding to the control data set in the key color setting register 212.

  The source selection register 214 is set with control data for designating source data to be superimposed with frame data in the overlay processor 130. As source data, either image data of a captured image from a camera module input via the camera I / F 110 or image data after decompression processing by the JPEG circuit 160 can be designated. Further, any one of three types of image data including image data input via the host I / F 120 may be designated as the source data. A source selection signal Ssel is output corresponding to the control data set in the source selection register 214.

  The path selection register 216 is set with control data for designating the output path of the image data after the overlay processing performed by the overlay processing unit 130. The output path of the image data after the superimposition processing includes a path for writing to the display memory via the YRC 190, or a path used for the compression processing of the JPEG circuit 160. In FIG. 1, when the image data after the superimposition process is used for the compression process of the JPEG circuit 160, it is written in the line buffer area LBR of the display memory 140 as a work area. It is used for processing. A path designation signal Psel is output corresponding to the control data set in the path selection register 216.

  Further, based on the setting data of the path selection register 216, the image data after the superimposition process may be written into the display memory 140, and the image data may be supplied to the compression process by the JPEG circuit 160. At this time, the image data written in the display memory 140 and the image data compressed by the JPEG circuit 160 are made the same. By doing so, it is possible to avoid the deterioration in usability for the user due to the fact that the image to be compressed is different from the display image of the display device.

  When the host accesses the start register 218, the start signal StartTrig for instructing the start of the overlay process becomes active.

  Hereinafter, a configuration example of the overlay processing unit 130 controlled based on the control data of the control register unit 210 will be described.

2. Superposition Processing Unit FIG. 4 is a block diagram showing a configuration example of the superposition processing unit 130 in the present embodiment. In FIG. 4, the relationship between the overlay processing unit 130 and the display memory 140 is schematically shown. The same parts as those in FIG.

  When the activation signal StartTrig from the control register unit 210 becomes active, the overlay processing unit 130 performs overlay processing.

  The overlay processing unit 130 includes an address generation unit 131, a read buffer 132, a data manager 133, a comparator 134, and selectors 135, 136, and 137.

  The address generation unit 131 generates a read address Adr for reading image data from the display memory 140. More specifically, the address generator 131 generates a read address for reading frame data (image data in a broad sense) from the frame data storage area FAR of the display memory 140.

  The read buffer 132 buffers the read data RdData read from the display memory 140. The read buffer 132 has a first-in first-out (FIFO) that is a memory having a first-in first-out function. When the write enable BufWrEn from the data manager 133 becomes active, read data RdData is written to the FIFO. When the read enable BufRdEn from the data manager 133 becomes active, data is read from the FIFO. When the data stored in the FIFO is full, the buffer full BufFull becomes active. When the data stored in the FIFO becomes empty, the buffer empty BufEmpty becomes active.

  The data manager 133 controls the synchronization of the reading operation of the two data that performs the superimposing process and controls the superimposing process of the two data. The data manager 133 controls reading of read data from the display memory 140 based on the buffer full BufFull and the buffer empty BufEmpty from the read buffer 132. Therefore, the data manager 133 outputs a read request RdReq to the display memory 140, and reads from the display memory 140 (more specifically, a memory controller (not shown) that controls access to the display memory 140; the same applies hereinafter) RdAck. Receive. The data manager 133 outputs a write enable BufWrEn to write the read data RdData read from the display memory 140 in response to the read request RdReq to the read buffer 132.

  The data manager 133 receives a start signal for an image selected by the source selection signal Ssel. That is, the selector 135 indicates a camera image start signal indicating the start timing of one vertical scanning period of the image input from the camera I / F 110 and the start timing of one vertical scanning period of the image after the expansion processing by the JPEG circuit 160. One of the decoded image start signals is selected and output based on the source selection signal Ssel. The selection output of the selector 135 is input to the data manager 133. The data manager 133 performs control to read out the frame data held in the read buffer 132 in synchronization with the image start signal of the source data selected by the source selection signal Ssel.

  FIG. 5 shows a flow of an example of frame data read control from the display memory 140 performed by the data manager 133 of FIG.

  First, it is determined whether or not the activation signal StartTrig from the control register unit 210 is active (step S300), and waits until the activation signal StartTrig becomes 1 (step S300: N).

  When the start signal StartTrig becomes 1 (step S300: Y), the read request RdReq for the display memory 140 is activated (set to 1), and the reading of frame data from the display memory 140 is requested (step S301).

  Next, when the read acknowledge RdAck from the display memory 140 is not active (step S302: N), the read request RdReq remains active (step S303), and the process returns to step S302.

  In step S302, when the read acknowledge RdAck from the display memory 140 is active (step S302: Y), the data manager 133 determines whether or not the buffer full BufFull from the read buffer 132 is active (step S304).

  When it is determined that the read buffer 132 is not full due to the buffer full BufFull (step S304: Y), whether the image end signal FrmEnd indicating the end of one vertical scanning period of the input image (camera image or decoded image) is 0 or not. Is determined (step S305).

  When the image end signal FrmEnd is not 0 (step S305: N), a series of processing ends (end). When the image end signal FrmEnd is 0 (step S305: Y), the process returns to step S301.

  When it is determined in step S304 that the read buffer 132 is full due to the buffer full BufFull (step S304: N), it is determined whether or not the read enable BufRdEn is 1 (step S306).

  When it is determined that the read enable BufRdEn is 1 (step S306: Y), the process proceeds to step S305. When it is determined that the read enable BufRdEn is not 1 (step S306: N), the read request RdReq is set to 0 (step S307), and the process returns to step S304.

  As described above, the data manager 133 controls reading of frame data from the display memory 140.

  FIG. 6 shows a timing chart of an operation example of read control by the data manager 133 of FIG.

  In FIG. 6, buffer data BufData indicates frame data written to the read buffer 132, and buffer out data BufOutData indicates frame data read from the read buffer 132.

  As described above, when the start signal StartTrig becomes active, the data manager 133 activates the read request RdReq and outputs the read address Adr, and inactivates the read request RdReq in accordance with the frame data read status from the display memory 140. The frame data is accumulated in the read buffer 132, and the frame data is sequentially read out. The read timing of each pixel of the frame data from the read buffer 132 is synchronized with the supply timing of each pixel of the image data of the input image.

  In FIG. 4, the key color designation signal Kc and the frame data read from the read buffer 132 are input to the comparator 134. The comparator 134 compares the data of each pixel of the frame data with the key color designation signal Kc and notifies the data manager 133 of the comparison result.

  A path designation signal Psel is input to the data manager 133. The data manager 133 performs selection control of the selectors 136 and 137 for each pixel based on the path designation signal Psel and the comparison result from the comparator 134.

  The selector 136 receives camera data, which is image data input via the camera I / F 110, and frame data read from the read buffer 132. The output of the selector 136 is supplied to the YRC 190. The camera data image start signal is a camera image start signal input to the selector 135. When it is designated by the path designation signal Psel to write the image data after the overlay processing into the image data storage area IAR of the display memory 140, and the comparison result signal of the comparator 134 indicates a mismatch, the selector 136 Output pixel data. Further, when it is designated by the path designation signal Psel to write the image data after the superimposition processing in the image data storage area IAR of the display memory 140, and the comparison result signal of the comparator 134 indicates coincidence, the selector 136 Output pixel data.

  The selector 137 is input with decode data that is image data after decompression and frame data read from the read buffer 132. The output of the selector 137 is written in the line buffer area of the display memory 140. The image start signal of the decoded data is a decoded image start signal input to the selector 135. When the image data after the overlay processing is designated to be subjected to compression processing by the path designation signal Psel and the comparison result signal of the comparator 134 indicates a mismatch, the selector 137 outputs the pixel data of the frame data. When the image data after the overlay processing is designated to be subjected to compression processing by the path designation signal Psel and the comparison result signal of the comparator 134 indicates coincidence, the selector 137 outputs the pixel data of the decoded data.

  As described above, the superimposition processing unit 130 performs superimposition processing using the frame data in which the key color is set, so that the increase in the amount of information to be stored is suppressed even when the image size increases. A matching process can be performed.

  Further, it is desirable to supply the read enable BufRdEn and the buffer empty BufEmpty to the interrupt generating unit 200. In this way, the interrupt generation unit 200 can perform an interrupt notification with an interrupt signal on the condition that an instruction to read frame data from the read buffer 132 is issued even though the read buffer 132 is empty. As a result, it is possible to avoid unnecessary superimposition processing without frame data.

  Note that the overlay processing unit 130 may adopt image data input from the host I / F 120 instead of the decoded data. In this case, the selector 135 receives a start signal of image data input from the host I / F 120 instead of the decoded image start signal.

  4 shows only the configuration for realizing the read control for the display memory 140, the overlay processing unit 130 has a write control circuit (not shown), and image data is stored in the display memory 140 by the write control circuit. Can write.

3. Example of Operation of Present Embodiment Prior to describing the operation of the image processing controller in this embodiment, a display controller as an image processing controller in a comparative example of this embodiment will be described. The display controller in the comparative example described below does not include the overlay processing unit 130 in the present embodiment.

  FIG. 7 shows a block diagram of the configuration of the display controller in the first comparative example of the present embodiment.

  The display controller 400 in the first comparative example performs an overlay process of camera data and frame data, compresses the processed image data, and outputs the compressed image data to the host.

  The display controller 400 includes a display memory 410, and frame data is stored in the display memory 410. Then, the overlay processing of the frame data and the camera data from the camera module 420 is performed. The display controller 400 performs compression processing on the image data after the overlay processing, and supplies the image data subjected to the compression processing to the host.

  That is, first, camera data of an image captured by the camera module 420 is stored in the display memory 410 via the resizer 430 and the YRC 440 (SEQ1). Then, the display controller 400 performs overlay processing using the camera data and frame data stored in the display memory 410, and generates overlay data that is image data after the overlay processing (SEQ2). This superimposed data is read again from the display memory 410 and supplied to the JPEG circuit 450 via the resizer 430. The JPEG circuit 450 performs compression processing on the overlay data, and outputs the compressed image data to the host 460 (SEQ3).

  As described above, in the first comparative example, since the camera data is temporarily stored in the display memory 410 and is subjected to compression processing, the display image on the LCD panel 470 and the image subjected to the compression processing by the JPEG circuit 450 are made the same. However, it cannot be generated in real time.

  FIG. 8 shows a block diagram of the configuration of the display controller in the second comparative example of the present embodiment. In FIG. 8, the same parts as those of FIG.

  The display controller 500 in the second comparative example performs an overlay process on the decoded data and the frame data, and displays the processed image data on the LCD panel 470.

  The display controller 500 includes a display memory 510, and the display memory 510 stores image data after the overlay process. The LCD panel 470 is driven based on the image data read from the display memory 510.

  Frame data is stored in the external memory 530 accessed by the host 520. First, the host 520 reads the compressed image data (decoded data) acquired as content data, and causes the JPEG circuit 450 of the display controller 500 to perform decompression processing (SEQ10). Then, the host 520 reads the image data after the decompression process, performs the overlay process with the frame data stored in the external memory 530, and generates the overlay data (SEQ11).

  Again, the host 520 causes the JPEG circuit 450 to perform compression processing on the overlay data and stores it in the external memory 530 (SEQ12).

  Thereafter, in accordance with the display timing of the LCD panel 470, the host 520 reads the superposed data subjected to the compression process from the external memory 530, performs the decompression process by the JPEG circuit 450, and then writes it in the display memory 510 ( SEQ13).

  As described above, in the second comparative example, since the decompression process is performed twice and the compression process is performed once on the content data, it cannot be processed in real time.

  In contrast to the comparative example as described above, the image processing controller 100 according to the present embodiment can perform superimposition processing in real time by providing the superimposition processing unit 130 in the previous stage of the display memory 140.

  FIG. 9 is an explanatory diagram of a first operation example of the image processing controller in the present embodiment. 9, the same parts as those in FIG. 1 or FIG. 4 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  In FIG. 9, the image data storage area IAR is accessed by the triple buffering method. Therefore, the image data storage area IAR includes first to third image data storage areas IAR1 to IAR3. Accordingly, the storage areas of the first to third image data storage areas IAR1 to IAR3 are selected in order, and the image data is written in the storage areas in order. Then, the image data read from the storage area different from the storage area where the image data is written is supplied to the LCD I / F 150. In addition, when performing rotation processing for rotating the orientation of the image represented by the image data read from the image data storage area IAR around the reference position of the image, it is possible to access one screen by using the triple buffering method. It is possible to avoid the occurrence of a phenomenon such as a minute image being interrupted.

  In the first operation example, frame data is set in the frame data storage area FAR of the display memory 140 via the host I / F 120.

  In this case, the frame data generated by the host is input to the host I / F 120. The frame data output from the host I / F 120 is converted into, for example, a YUV format by the RYC 170, and then the image size is converted by the resizer 180. Thereafter, the YRC 190 does not perform format conversion and writes the data as it is in the frame data storage area FAR of the display memory 140.

  FIG. 10 is an explanatory diagram of a second operation example of the image processing controller in the present embodiment. 10, the same parts as those in FIG. 9 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  In the second operation example, a process of superimposing the camera data and the frame data input via the camera I / F 110 is performed. Then, the image data after the overlay processing is output to the LCD panel via the LCD I / F 150.

  First, camera data input via the camera I / F 110 is input to the selector 136 of the overlay processing unit 130 after the image size is changed by the resizer 180. The camera I / F 110 outputs a camera image start signal to the overlay processing unit 130 based on the camera data.

  The data manager 133 accumulates frame data from the frame data storage area FAR of the display memory 140 in the read buffer 132, and reads the frame data from the read buffer 132 in accordance with the input timing of each pixel of camera data. At this time, the data manager 133 reads out the frame data with reference to the timing defined by the camera image start signal.

  Then, while performing the key color processing described above, the selector 136 outputs the image data after the overlay processing. The image data is converted into the RGB format by the YRC 190 and then written in any of the first to third image data storage areas IAR1 to IAR3 in the image data storage area IAR of the display memory 140. Then, the image data read from the storage area different from the storage area where the image data is written is supplied to the LCD I / F 150.

  As described above, in the second operation example, the camera data can be superimposed in real time, and the processing result can be written in the display memory 140. Therefore, the LCD panel is driven based on the image data superimposed on the camera data in real time.

  FIG. 11 is an explanatory diagram of a third operation example of the image processing controller in the present embodiment. 11, the same parts as those in FIG. 9 are denoted by the same reference numerals, and the description thereof is omitted as appropriate.

  In the third operation example, the overlapping process of the decoded data and the frame data that have been decompressed by the JPEG circuit 160 is performed. Then, the image data after the overlay processing is output to the LCD panel via the LCD I / F 150.

  First, the host sets the image data subjected to the compression process in the compression / decompression process image data storage area JAR of the display memory 140 via the host I / F 120. Thereafter, the JPEG circuit 160 reads out the image data from the image data storage area JAR for compression / decompression processing, performs the decompression processing, and writes the image data after the decompression processing in the line buffer area LBR of the display memory 140.

  The image data written in the line buffer area LBR of the display memory 140 is sequentially read and the image size is converted by the resizer 180. The image data whose image size has been converted is input to the selector 136 of the overlay processing unit 130. The resizer 180 outputs a decoded image start signal to the superposition processing unit 130 based on the decompressed image data (decoded data).

  The data manager 133 accumulates the frame data from the frame data storage area FAR of the display memory 140 in the read buffer 132, and reads the frame data from the read buffer 132 in accordance with the input timing of each pixel of the decoded data. At this time, the data manager 133 reads out the frame data with reference to the timing defined by the decoded image start signal.

  Then, while performing the key color processing described above, the selector 136 outputs the image data after the overlay processing. The image data is converted into the RGB format by the YRC 190 and then written in any of the first to third image data storage areas IAR1 to IAR3 in the image data storage area IAR of the display memory 140. Then, the image data read from the storage area different from the storage area where the image data is written is supplied to the LCD I / F 150.

  As described above, in the third operation example, decoding processing can be performed in real time on the decoded data, and the processing result can be written in the display memory 140. Therefore, the LCD panel is driven based on the image data superimposed on the decoded data in real time.

  In the third operation example, the JPEG circuit 160 performs the overlay process on the image data that has undergone the decompression process, but the image data may be directly supplied via the host I / F 120. In this case, an input image start signal of image data input via the host I / F 120 is input to the selector 135 of the overlay processing unit 130. Therefore, the data manager 133 can read frame data from the read buffer 132 in accordance with the input timing of each pixel of the image data directly input via the host I / F 120.

  FIG. 12 is an explanatory diagram of a fourth operation example of the image processing controller in the present embodiment. In FIG. 12, the same parts as those of FIG.

  In the fourth operation example, a process of superimposing camera data and frame data input via the camera I / F 110 is performed. Then, compression processing is performed on the image data after the overlay processing, and the data is output from the host I / F 120 to the host. It is also possible to output the same image data as the image data subjected to the compression process to the LCD panel via the LCD I / F 150.

  First, camera data input via the camera I / F 110 is input to the selector 137 of the overlay processing unit 130 after the image size is changed by the resizer 180. The camera I / F 110 outputs a camera image start signal to the overlay processing unit 130 based on the camera data.

  The data manager 133 accumulates frame data from the frame data storage area FAR of the display memory 140 in the read buffer 132, and reads the frame data from the read buffer 132 in accordance with the input timing of each pixel of camera data. At this time, the data manager 133 reads out the frame data with reference to the timing defined by the camera image start signal.

  Then, while performing the key color process described above, the selector 137 outputs the image data after the overlay process. The output of the selector 137 is written in the line buffer area LBR of the display memory 140. The JPEG circuit 160 reads the image data from the line buffer area LBR, performs compression processing, and writes the compressed image data in the compression / expansion processing image data storage area JAR of the display memory 140. The image data written in the compression / decompression processing image data storage area JAR of the display memory 140 is output to the host via the host I / F 120.

  At the same time as the selector 137, the selector 136 also outputs the image data after the superimposition process, converts it to the RGB format by the YRC 190, and then stores the first to third image data in the image data storage area IAR of the display memory 140. It may be written in any of the areas IAR1 to IAR3. In this case, image data read from a storage area different from the storage area in which the image data is written is supplied to the LCD I / F 150.

  As described above, in the fourth operation example, it is possible to perform superimposition processing on camera data in real time, compress the processing result, supply it to the host, and store it. Therefore, the image data that is superimposed on the camera data can be directly compressed and stored. When this image data is displayed on the LCD panel, the image data of the image displayed on the LCD panel can be compressed and stored.

5. Electronic Device FIG. 13 is a block diagram showing a configuration example of an electronic device according to this embodiment. Here, a block diagram of a configuration example of a mobile phone is shown as an electronic device.

  The cellular phone 700 includes a display controller 710 that functions as the image processing controller of FIG. The mobile phone 700 includes a camera module 720. The camera module 720 includes a CCD camera, and supplies image data captured by the CCD camera to the display controller 710.

  The mobile phone 700 includes a display panel (an electro-optical device in a broad sense, a display device in a broader sense) 730. An LCD panel can be used as the display panel 730. In this case, the display panel 730 is driven by the display driver 740. The display panel 730 includes a plurality of scanning lines, a plurality of data lines, and a plurality of pixels. The display driver 740 has a function of a scanning driver that selects a scanning line in units of one or a plurality of scanning lines, and also has a function of a data driver that supplies a voltage corresponding to pixel data to the plurality of data lines.

  The display controller 710 is connected to the display driver 740 and supplies RGB format image data to the display driver 740.

  The host 750 is connected to the display controller 710. The host 750 controls the display controller 710. Further, the host 750 can supply communication data including image data received via the antenna 760 to the display controller 710 after demodulating by the modem 770. The display controller 710 causes the display panel 730 to display the display driver 740 based on the image data.

  The host 750 can instruct transmission to another communication apparatus via the antenna 760 after the image data generated by the camera module 720 is modulated by the modulation / demodulation unit 770.

  The host 750 performs transmission / reception processing of image data, imaging of the camera module 720, and display processing of the display panel based on operation information from the operation input unit 780.

  In FIG. 13, the LCD panel is described as an example of the display panel 730, but the display panel 730 is not limited to this. The display panel 730 may be an electroluminescence or plasma display device, and can be applied to a display controller that supplies image data to a display driver that drives them. The display controller 710 may output YUV format image data to a CRT device connected via an output terminal (not shown).

  The present invention is not limited to the above-described embodiment, and various modifications can be made within the scope of the gist of the present invention.

  In the invention according to the dependent claims of the present invention, a part of the constituent features of the dependent claims can be omitted. Moreover, the principal part of the invention according to one independent claim of the present invention can be made dependent on another independent claim.

FIG. 3 is a block diagram of a configuration example of an image processing controller in the present embodiment. FIG. 2 is a schematic explanatory diagram of an overlay process of an overlay processor in FIG. 1. The figure which shows the outline | summary of a structure of the control register part of FIG. The block diagram of the structural example of the superimposition process part in this embodiment. FIG. 5 is a flowchart showing an example of frame data read control performed by the data manager of FIG. 4. FIG. 5 is a timing diagram of an operation example of read control by the data manager of FIG. 4. The block diagram of the structure of the display controller of the 1st comparative example of this embodiment. The block diagram of the structure of the display controller of the 2nd comparative example of this embodiment. Explanatory drawing of the 1st operation example of the image processing controller in this embodiment. Explanatory drawing of the 2nd operation example of the image processing controller in this embodiment. Explanatory drawing of the 3rd operation example of the image processing controller in this embodiment. Explanatory drawing of the 4th operation example of the image processing controller in this embodiment. 1 is a block diagram of a configuration example of an electronic device according to an embodiment.

Explanation of symbols

100 image processing controller, 110 camera I / F, 120 host I / F,
130 overlay processing unit, 131 address generation unit, 132 read buffer,
133 data manager, 134 comparator, 135, 136, 137 selector,
140 display memory, 150 LCD I / F, 160 JPEG circuit,
170 RYC, 180 Resizer, 190 YRC, 200 Interrupt generator,
210 control register section, 212 key color setting register,
214 source selection register, 216 path selection register, 218 activation register,
Adr read address, BufEmpty buffer empty, BufFull buffer full,
BufRdEn read enable, BufWrEn write enable,
FAR frame data storage area, IAR image data storage area,
JAR image data storage area for compression / decompression processing, Kc key color designation signal,
LBR line buffer area, Psel path designation signal, RdData read data,
RdReq read request, RdAck read acknowledge, Ssel source selection signal,
StartTrig start signal

Claims (13)

An image processing controller for superimposing two images,
An image data input interface through which image data of the input image is input;
An overlay processing unit that performs overlay processing of frame data and image data of the input image to display a background image;
A display memory in which image data after the overlay processing performed by the overlay processing unit is written;
An image data output interface for outputting image data read from the display memory,
The frame data is
The image data of a frame image in which an area for superimposing the input image in the background image is designated,
The image processing controller, wherein the superimposition processing unit performs a superimposition process for superimposing the input image on the area. An image processing controller for superimposing two images,
A camera interface to which image data of a captured image is input;
A host interface for inputting image data from the host;
An overlay processing unit that performs overlay processing of frame data for displaying a background image and image data of an input image input via the camera interface or the host interface;
A display memory in which image data after the overlay processing performed by the overlay processing unit is written;
An image data output interface for outputting image data periodically read from the display memory,
The frame data is
The image data of a frame image in which an area for superimposing the input image in the background image is designated,
The image processing controller, wherein the superimposition processing unit performs a superimposition process for superimposing the input image on the area. In claim 1 or 2,
Including an image compression / decompression unit that performs decompression processing of the image data subjected to compression processing,
The overlay processing unit
The image data after the decompression process performed by the image compression / decompression unit and the frame data are superimposed.
An image processing controller, wherein the image data after the superimposition processing is written in the display memory. In claim 3,
A source selection register for designating the image data of the captured image or the image data after the decompression process as source data to be superimposed with the frame data;
And a path selection register for designating whether the image data after the superimposition processing is written in the display memory or supplied to the compression processing by the image compression / decompression unit. Processing controller. In claim 4,
Based on the setting data of the path selection register, the image data after the overlay process is written to the display memory, and the image data is supplied to the compression process by the image compression / decompression unit,
An image processing controller, wherein the image data written in the display memory and the image data compressed by the image compression / decompression unit are the same. In any one of Claims 1 thru | or 5,
A frame data storage unit for storing the frame data;
Including an interrupt generation unit for generating an interrupt signal,
The overlay processing unit
A read buffer for buffering the frame data read from the frame data storage unit;
An image processing controller, wherein the interrupt generation unit generates the interrupt signal on the condition that an instruction to read frame data from the read buffer is issued even though the read buffer is empty. In any one of Claims 1 thru | or 6.
The frame data is
Image data of a frame image in which a key color is set in a region where the input image or the image data of the decompressed image data is superimposed;
The overlay processing unit
An image processing controller, wherein image data of a pixel in an area where the key color is set in the frame data is replaced with image data of the input image or image data after the decompression process. A display device;
An image processing controller according to any one of claims 1 to 7,
An electronic device comprising: a display driver that drives the display device based on image data supplied by the image processing controller. An image processing method for superimposing two images,
Set the frame data for displaying the background image in the frame data storage unit,
While reading the frame data from the frame data storage unit in synchronization with the image data of the captured image, the image data of the captured image and the frame data are overlaid,
Write the image data after the overlay process to a display memory,
The image data read from the display memory is periodically read,
The frame data is
An image processing method, wherein an area in which an input image is superimposed on the background image is image data of a specified frame image. An image processing method for superimposing two images,
Set the frame data for displaying the background image in the frame data storage unit,
The decompressed image data and the frame data are superimposed while reading out the frame data from the frame data storage unit in synchronization with the decompressed image data obtained by decompressing the compressed image data. Process
Write the image data after the overlay process to a display memory,
The image data read from the display memory is periodically read,
The frame data is
An image processing method, wherein an area in which an input image is superimposed on the background image is image data of a specified frame image. In claim 10,
The image data after the overlay process is written to the display memory, and the image data is compressed.
An image processing method, wherein the image data written to the display memory and the image data to be compressed are the same. In claim 10 or 11,
An interrupt notification is performed under a condition that an instruction to read frame data from the read buffer is issued even though a read buffer for buffering the frame data read from the frame data storage unit is empty. An image processing method. In any of claims 10 to 12,
The frame data is image data of a frame image in which a key color is set in a region where the input image or the image data of the decompressed image data is superimposed,
An image processing method, wherein image data of pixels in an area where the key color is set in the frame data is replaced with image data of the input image or image data after the decompression process.

Images