JP2015099443A - Image processing apparatus, image processing method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the drawing time in drawing a second image over a first image, and reduce the band width required for the output of drawn data.SOLUTION: An image processing apparatus specifies an area to be updated in an area of a frame immediately before the current frame according to a drawing command; reads out data of the specified area from coded image data of the frame immediately before the current frame, decodes the read-out data, and overwrites the data of the area to be updated in the current frame on the decoded data and encodes the resulting data to create partially-coded image data; and merges the coded image data of the frame immediately before the current frame and the partially-coded image data with reference to the area to be updated in the area of the frame immediately before the current frame.

Description

  The present invention relates to an image processing apparatus, an image processing method, and a computer program, and is particularly suitable for use in drawing an image by overwriting a second image on a first image.

  Systems that use vector graphics for drawing animation have become widespread. With the increase in screen size and resolution of display devices, the number of pixels handled by the renderer used to draw an image is expected to increase in the future. If the number of pixels to be drawn is large, vector graphics that specify the shape of a drawing figure (object) using coordinate information and color information are used for drawing instructions compared to raster graphics that use bitmaps. Is small.

  A renderer that draws vector graphics animation draws each frame constituting the animation in turn. As a method of drawing a frame, there is a method of drawing a difference updated in the current frame in a previously drawn image in addition to a method of drawing all animation elements included in each frame every frame. Patent Document 1 discloses a method of overwriting an already drawn frame image with only an area updated by the operation of an object.

  In a system that draws an arbitrary vector graphics animation, the time taken to render the animation by the renderer cannot be predicted. Some of such systems employ a configuration using a double buffer or a multi-buffer in order to facilitate the timing control of the rendered image display process and the image update process. In a system using a double buffer, when the renderer draws a difference from the previous frame, it is necessary to copy appropriate data between the buffers constituting the double buffer. Draw the color information necessary for drawing the current object by referring to the case where the frame being drawn overlaps a semi-transparent object over the object drawn in the previous frame, or any place in the drawn frame based on the drawing command There are cases where it is obtained from a completed frame. For this reason, the drawing result of the previous frame is copied to the buffer or work memory to be drawn, and the drawing process is started after the copying is completed. In order to reduce the waiting time required to copy an image between buffers performed prior to the drawing process, Patent Document 2 discloses that an area that does not change is copied on-screen and a difference drawing process is performed in parallel. Are disclosed.

JP 2004-265331 A JP 2010-164972 A

However, with the increase in the number of drawing pixels, the bandwidth required for outputting the image data that the renderer refers to at the time of drawing and the drawn data has increased. In a system using a double buffer or a multi-buffer, an additional bandwidth is required for copying a drawn image and a background image.
In the method of redrawing all the drawing elements, the bandwidth required for copying between frames is not necessary. However, in cases where bitmap data is referenced or modified in an area where there is no difference between frames, it is necessary to repeat access to a plurality of input bitmap data every frame. This increases the bandwidth for redrawing the frame. In addition, the time required for the drawing process for each frame also increases.
The present invention has been made in view of such problems, and it is necessary to reduce the drawing time when drawing the second image over the first image and to output the drawn data. The purpose is to reduce the bandwidth.

  The image processing apparatus according to the present invention provides an update area updated by the second image among the areas of the first image, based on a drawing instruction of the second image input after the first image. Setting means for setting and decoding the code image data of the first image in the update area among the code image data of the first image stored in the storage medium, and the update area of the first image A decoding means for generating decoded data in the above; and a drawing process for overwriting the second image on the decoded data in the update area of the first image generated by the decoding means; A drawing unit that generates drawing data of an area including the update region of the image and outputs drawing data of the update region of the second image, and the update region of the second image output by the drawing unit In Encoding means for encoding the drawing data, and encoded image data in an area different from the update area of the first image stored in the storage medium, and the encoding means encoded by the encoding means A code image synthesizing unit that synthesizes drawing data in the update area of the second image and generates code image data of the second image; and a second image generated by the code image synthesizing unit. Storage means for storing code image data in the storage medium.

  According to the present invention, it is possible to reduce the drawing time when drawing the second image over the first image and to reduce the bandwidth necessary for outputting the drawn data.

It is a block diagram which shows the structure of an image processing apparatus. It is a figure which shows a drawing command. It is a figure which shows the image of 2 frames drawn based on a drawing command. It is a figure which shows the positional relationship of the bounding box of an object. It is a figure which shows an example of update area information. It is a figure which shows the result of having decoded the update area | region of the 1st frame. It is a figure which shows the result of drawing of the update area | region in a 2nd frame. It is a figure which shows partial code | cord image data. It is a figure which shows the code image data of a 1st frame. It is a figure which shows area | region determination data. It is a flowchart which shows operation | movement of a code image synthetic | combination part. It is a figure which shows the code image data of a 2nd frame. It is a figure which shows the other example of update area information. It is a figure which shows the other example of update area information.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram illustrating an example of the configuration of the image processing apparatus 100 according to the present embodiment.
The image input unit 101 receives a vector graphics drawing command. The update area extraction unit 102 extracts, from the drawing command received by the image input unit 101, information about the drawing result area of the previous frame that is referred to during the drawing process of the current frame as update area information. The drawing result area one frame before that is referred to during the drawing process of the current frame is an area that is updated in the current frame among the drawing result areas one frame before. In the following description, this area is referred to as an update area as necessary.

The partial image decoding unit 103 partially decodes (decodes) the drawing result of the previous frame. The drawing unit 104 overwrites the update region data of the current frame on the update region data partially decoded by the partial image decoding unit 103, draws the update region, and generates drawing data. The encoding unit 105 encodes (encodes) the drawing data generated by the drawing unit 104.
The code image synthesis unit 106 merges (synthesizes) the code image data of the previous frame stored in the data storage unit 108 and the drawing data encoded by the encoding unit 105 to code the current frame. Generate image data. The output unit 107 outputs the code image data obtained by the code image synthesis unit 106 to the data storage unit 108. The decoding unit 109 decodes the code image data stored in the data storage unit 108 to generate decoded image data. The display unit 110 displays an image based on the decoded image data generated by the decoding unit 109.

Hereinafter, an example of these operations will be described.
The image processing apparatus 100 converts image data described in SVG or JAVA (registered trademark) into a drawing command including a frame drawing completion instruction using an appropriate parser based on an operation instruction by a user, and inputs an image. Input to the unit 101.
FIG. 2 is a diagram illustrating an example of a drawing command. FIG. 3 is a diagram conceptually showing an example of a two-frame image drawn based on the drawing command shown in FIG. Specifically, FIG. 3A shows an image of the first frame (first image), and FIG. 3B shows an image of the second frame displayed after the first frame (first image). 2 image).
Command 201 to command 204 are drawing commands for the first frame image shown in FIG. Command 205 to command 215 are drawing commands for the image of the second frame shown in FIG. In the present embodiment, an example in which an image of the second frame is drawn by overwriting an additional object on the image of the first frame will be described as an example. In the present embodiment, the X coordinate increases to the right and the Y coordinate increases upward unless otherwise specified.

The command 201 indicates that the entire frame to be drawn is filled with a color designated as an argument and initialized.
Command 202 indicates changing the setting of the fill color. When the fill (FILL) is instructed after the command 202, the value specified by the argument of the command 202 is used. Hereinafter, it is assumed that the 8-bit color information of RGBA is used in the instruction for changing the color setting included in FIG. That is, the instructions 202, 205, 209, and 213 in FIG. 2 are represented by 32-bit data in which Red, Green, Blue, and α values are packed in the order of 8 bits. The color set by the instruction 202 is opaque gray (0xAAAAAAFF).

The command 203 indicates that the coordinates of the lower left and upper right corners of the rectangle are received, and the rectangle 302 filled with gray set by the command 202 is drawn. In the example shown in FIG. 2, the lower left and upper right coordinates of the rectangle are (X, Y) = (2.0, 1.0), (5.0, 3.0), respectively.
The command 204 indicates that the drawing completion of the first frame is instructed.
Command 205 indicates changing the fill color to transparent white (0xFFFFFF00).
The command 206 draws a rectangle having coordinates (X, Y) = (6.2, 6.0), (7.8, 8.5) as the lower left and upper right, respectively, with the values set by the command 205. Since the α value is 0 (zero), this rectangle does not appear in the drawing result of FIG.

Command 207 indicates that the fill color is set to translucent red (0xFF000080).
The command 208 indicates that an ellipse is drawn by designating the center, the width and height, and the angle. Specifically, the command 208 is such that the center coordinates are (X, Y) = (4.0, 4.0), the width and height are 2.0, the drawing start start angle is 0 (zero) degrees, and the drawing end angle is 360 degrees. Indicates that the circle 303 is drawn with a solid color.
Command 209 indicates to set the fill color to translucent blue (0x0000FF80).
An instruction 210 indicates coordinate information used by the instruction 211.

In the command 211, MOVETO that specifies the drawing start coordinates, LINETO that connects the coordinates specified from the current coordinates with line segments, and CLOSEPATH that connects the current coordinates and the drawing start coordinates are specified. In the processing included in the command 211, the coordinates given by the command 210 are sequentially referred to.
The command 212 indicates that the triangle 304 designated by the commands 210 and 211 is drawn.
The command 213 indicates that the fill color is set to black (0x000000FF).
The command 214 indicates that a square 305 having the coordinates (X, Y) = (5.1, 2.1), (5.9, 2.9) as the lower left and upper right, respectively, is drawn.
The command 215 indicates that the drawing completion of the second frame is instructed.
The drawing command input to the image input unit 101 is sent to the drawing unit 104 and the update area extraction unit 102.

The update area extraction unit 102 extracts a graphic drawing command related to drawing of a graphic from the drawing command, and generates update area information from the coordinate information given to the extracted graphic drawing command.
Hereinafter, in the present embodiment, when the image of the second frame shown in FIG. 3B is drawn, a method of generating update area information using a bounding box that is a rectangular area surrounding an area where an object may exist Will be described as an example.

FIG. 4 is a diagram illustrating an example of a positional relationship on the logical coordinates of the bounding box of each object drawn by the commands 205 to 215.
The rectangular bounding box 401 drawn by the commands 205 and 206 in FIG. 2 has coordinates (X, Y) = (6.2, 6.0), (7.8, 8.5) based on the information on the vertices constituting the rectangle. ) And a rectangle.
The bounding box 402 of the circle 303 drawn by the instructions 207 and 208 has the coordinates of the lower left and upper right from the information of the center point of the circle and the information of the width and height, respectively (X, Y) = (2.0, 2.0) , (6.0, 6.0).

The bounding box 403 of the triangle 304 drawn by the instructions 209 to 212 has (X, Y) = (2.0, 0.0), (3.0, 2.0) as the coordinates of the lower left and upper right from the information on the vertices constituting the triangle. It becomes a rectangle.
The bounding box 404 of the square 305 drawn by the instructions 213 to 215 has the coordinates of the lower left and upper right as (X, Y) = (5.1, 2.1), (5.9, 2.9) from the information on the vertices constituting the square, respectively. It becomes a square.

Actually, in order to perform drawing with pixel accuracy (unit of one pixel), the update region extraction unit 102 extracts “one pixel or a set of a plurality of pixels” including the bounding box as update region information.
FIG. 5 is a diagram illustrating an example of the update area information. Specifically, FIG. 5A is a diagram illustrating an example of an update region in which the bounding boxes 401 to 404 illustrated in FIG. 4 are mapped to a two-dimensional coordinate system with pixel accuracy of 8 pixels in the vertical direction and 10 pixels in the horizontal direction. FIG. 5B is a diagram showing an example of update area information generated from the update area shown in FIG.

An area 501 is an area including the bounding box 401. In the area 501, the start position of the object update area is (X, Y) = (6, 6) as shown in the entry 513 in FIG. It is defined as an area of 2 pixels. Different IDs are assigned to the objects recorded in the update area information as identification information for uniquely identifying the drawing order of the objects.
Similarly, the region 502 is a region including the bounding box 402, the region 503 is a region including the bounding box 403, and the region 504 is a region including the bounding box 404. Area 502, area 503, and area 504 are stored in entry 511, entry 510, and entry 513, respectively.

Returning to the description of FIG. 1, the partial image decoding unit 103 reads out and decodes the code image data of the update region indicated by the update region information from the code image data of the previous frame placed in the data storage unit 108. The decoded result is output to the drawing unit 104.
The partial image decoding unit 103 reads the code image data of the first frame and decodes the portion corresponding to the coordinates of the update area information. In the present embodiment, a case where run-length compression is used for generating code image data will be described as an example. In the case of run length compression, the partial image decoding unit 103 scans the run length of the encoded first frame, decodes the region corresponding to the update region information, and outputs the decoded result to the drawing unit 104.

FIG. 6 is a diagram conceptually illustrating an example of a result of decoding the update area of the first frame. Specifically, FIG. 6 shows that the partial image decoding unit 103 decodes the part of the update area indicated by the update area information shown in FIG. 5B in the encoded image data of the first frame. It is a figure which shows an example of the pixel image obtained.
The area specified by the entry 510 in FIG. 5 is an area of the pixels 601 and 602. The area specified by the entry 512 is the area of the pixel 604. Similarly, the area specified by the entry 511 is an area having a width of 4 pixels and a height of 4 pixels with the pixel 603 being lower left, and the area specified by the entry 513 is an area having a width of 2 pixels having the pixel 605 being lower left. , A region with a height of 2 pixels.

  The decoding result (decoded data) shown in FIG. 6 may be placed in a work memory (not shown) that can be referred to by both the partial image decoding unit 103 and the drawing unit 104. When the decoding result (decoded data) is once stored in the work memory, the partial image decoding unit 103 sends the update area information ID and a pointer to the decoding result (decoded data) corresponding to each ID to the drawing unit 104. Output.

The drawing unit 104 sequentially executes drawing commands for the update area portion among the drawing commands for the second frame, and draws the update area for the second frame. When the drawing result of one frame before is required for the calculation of the pixel value, the drawing unit 104 refers to the decoded data output from the partial image decoding unit 103.
In the example shown in FIG. 2, the drawing result of one frame before is referred to in a pixel that meets one of the following two conditions.
The first is an area in which the fill color is designated as a translucent color, and the color of the previous frame needs to be superimposed on the base of the translucent color. In the example shown in FIG. 3, in drawing the circle 303 and the triangle 304, it is necessary to refer to the drawing data one frame before (first frame) for all the pixels where the object exists.

  The second is an area where an object less than one pixel is drawn. In FIG. 3, the area of the square 305 object drawn in the current second frame is 0.8 pixels wide and 0.8 pixels high, which is smaller than 1 pixel. Therefore, in the calculation of the color within one pixel, the color of the portion that is not covered by the object within one pixel and the color of the portion that is covered by the object are mixed, so it is necessary to refer to the pixel value one frame before.

FIG. 7 is a diagram illustrating an example of a rendering result (partial rendering result) of the update area (by the rendering unit 104) in the second frame.
In pixel 701, triangle 304 drawn in the second frame covers 75% of the pixel, and the remaining 25% is not drawn in the second frame. For pixel value calculation, the pixel coverage needs to be 100%. For this reason, the pixel color of the first frame drawn one frame before is used for the 25% area that is not drawn in the second frame. The value of the pixel 701 is a value obtained by mixing 25% of the background color and 75% of the translucent blue (0x0000FF80) of the triangle 304 at a ratio of 75%.
Similarly, the value of the pixel 702 is a value obtained by mixing 75% of the gray color (0xAAAAAAFF) of the rectangle 302 and 25% of the semi-transparent blue color (0x0000FF80) of the triangle 304.

The pixels 703, 704, 705, and 706 are included in the region 502, but the color of the circle 303 drawn by the drawing unit 104 does not contribute to the value of the pixel 703. Accordingly, the pixel values (decoding results shown in FIG. 6) obtained as the drawing result of the first frame are used as the values of the pixels 703, 704, 705, and 706 as they are.
The square 305 drawn in the second frame is opaque, but the square 305 covers only 81% of the pixels. Therefore, in order to calculate the contribution to the pixel value from the remaining 19% area, the drawing unit 104 determines the pixel value obtained as the drawing result of the first frame when determining the value of the pixel 707. Refer to
Note that the drawing unit 104 may execute all drawing commands for the second frame in order, perform all drawing for the second frame, and output only drawing data for the update area.

Returning to the description of FIG. 1, the encoding unit 105 generates partial code image data that is data obtained by encoding the drawing data drawn by the drawing unit 104 and outputs the partial code image data to the code image synthesis unit 106.
FIG. 8 is a diagram illustrating an example of partial code image data. Specifically, FIG. 8 is a diagram illustrating an example of partial code image data obtained by encoding the partial drawing result illustrated in FIG. 7 by the encoding unit 105. In this embodiment, in order to perform run length compression, the partial code is composed of a pixel value and a run length.
Entry 801 in FIG. 8 shows the result of encoding pixel 701 in FIG. 7, entry 802 in pixel 702, entry 803 in pixel 703, entry 804 in pixels 708 and 709, entry 805 in pixel 704, and entry 806 in pixel 707. . Similarly, encoding is performed in order from the smallest coordinate value.

Returning to the description of FIG. 1, the code image synthesis unit 106 merges the code image data of the first frame read from the data storage unit 108 and the partial code image data of the second frame output from the encoding unit 105. . Data to be selected at the time of merging is determined with reference to the update area information shown in FIG.
FIG. 9 is a diagram illustrating an example of the code image data of the first frame. Specifically, FIG. 9A is a diagram illustrating an example of an image with pixel accuracy when the first frame is drawn with a width of 10 pixels and a height of 8 pixels. FIG. 9B is a diagram illustrating an example of the code image data of the first frame. According to the drawing command of FIG. 2, Color0 is white and Color1 is 0xAAAAAAFF (gray).
Since the entry 910 is the result of encoding the background color region, which is a combination of 10 pixels of the scan line Y = 0 to 10 pixels and 2 pixels of X = 0 and X = 1 of the scan line Y = 1. The run length is 12, and the pixel value is Color0. Similarly, since the entry 911 is a gray region in which five pixels continue from the pixel 902, the run length is 5, and the pixel value is Color1.

Next, the code image synthesis unit 106 merges the code image data of the first frame (see FIG. 9B) and the partial code image data of the second frame (see FIG. 8), An example of the process for constructing the code image data of the second frame will be described.
The code image synthesizing unit 106 copies (applies) the portion for copying (applying) the code image data of the first frame and the partial code image data of the second frame from the update area information shown in FIG. ). In this embodiment, for the sake of simplification, the code image synthesis unit 106 reads the update area information and determines the copy source data in the pixel scanning order from (X, Y) = (0, 0). It is assumed that data is output in an intermediate format by applying to the area determination data shown.
Each entry in FIG. 10 has a source and a length. The source determines whether the copy source of the encoded image data of the second frame is the encoded image data of the first frame or the partially encoded image data of the second frame output from the encoding unit 105. Show. The length indicates how many consecutive pixel values obtained from each source.

FIG. 11 is a flowchart illustrating an example of the operation of the code image synthesis unit 106.
In the present embodiment, the code image synthesis unit 106 uses a pixel counter for determining the read condition of the code image data of the first frame. In step S1101, the code image composition unit 106 resets the value of the pixel counter to 0 (zero).
Next, in step 1102, the code image synthesis unit 106 determines whether there is an unprocessed entry in the area determination data illustrated in FIG. 10. If there is no unprocessed entry in the area determination data shown in FIG. 10 as a result of this determination, the processing according to the flowchart of FIG. 11 is terminated.

On the other hand, if there is an unprocessed entry in the area determination data shown in FIG. 10, the process proceeds to step S1103. In step S1103, the code image composition unit 106 reads out one entry of the area determination data shown in FIG.
In step S1104, the code image synthesis unit 106 determines whether the source value of the entry of the area determination data read in step S1103 is Frame2. As a result of the determination, if the source value of the entry of the area determination data is Frame1, the process proceeds to step S1105. On the other hand, if the source value of the area determination data entry is Frame2, the process proceeds to step S1110 described later.

In step S1105, the code image composition unit 106 determines whether the value of the pixel counter is 0 (zero). If the result of this determination is that the value of the pixel counter is 0 (zero), steps S1106 and S1107 are omitted, and processing proceeds to step S1108, which will be described later. On the other hand, when the value of the pixel counter is not 0 (zero) (positive), the process proceeds to step S1106.
In step S1106, the code image synthesis unit 106 reads out one entry of the code image data of the first frame Frame1 (see FIG. 9B).
Next, in step S1107, the code image composition unit 106 sets the run length of the entry of the code image data read in step S1106 as the value of the pixel counter (see FIG. 9B). Also, the code image composition unit 106 sets the pixel value of the entry of the code image data read out in step S1106 to the Frame1 output pixel value (see FIG. 9B).

Next, in step S1108, the code image composition unit 106 performs the following processing. That is, the frame 1 output pixel value set in step S1107 is set as the pixel value, the entry length of the area determination data read in step S1103 is set as the run length, and one entry of the encoded image data of the second frame Frame 2 is output. FIG. 12 is a diagram illustrating an example of the code image data of the second frame. When the process proceeds from step S1105 to step S1108, the code image synthesis unit 106 performs the following process. That is, one entry of the coded image data of the second frame Frame 2 is output with the Frame 1 output pixel value set last time in Step S 1107 as the pixel value and the pixel counter value as the run length.
Next, in step S1109, the code image composition unit 106 updates the value of the pixel counter by subtracting the length value of the entry of the area determination data read in step S1103 from the current value of the pixel counter. Thus, the process for one entry of the area determination data is completed, and the process returns to step S1102 to repeat the process.

As described above, if the source value of the entry of the area determination data is Frame 2 in step S1104, the process proceeds to step S1110. In step S1110, the code image synthesis unit 106 reads one entry of partial code image data of the second frame Frame2 (see FIG. 8).
Next, in step S <b> 1111, the code image synthesis unit 106 sets the pixel value of the entry of the partial code image read in step S <b> 1110 as the Frame2 output pixel value.
Next, in step S1112, the code image composition unit 106 performs the following processing. That is, one entry of the encoded image data of the second frame Frame 2 is output with the Frame 2 output pixel value set in step S1110 as the pixel value and the length of the entry of the area determination data read out in step S1103 as the run length.

When the code image data entry of the second frame Frame2 is output in steps S1110 to S1112, reading of the code image data entry of the first frame Frame1 in the area corresponding to the entry is omitted in steps S1113 to S1122.
Specifically, in step S1113, the code image synthesis unit 106 sets the length of the entry of the area determination data read in step S1103 as the skip length.
Next, in step S <b> 1114, the code image synthesis unit 106 determines whether or not the value of the pixel counter is greater than or equal to the skip length value. If the result of this determination is that the pixel counter value is greater than or equal to the skip length value, the skip process is completed with the code image data of the first frame Frame1 being processed. In this case, in step S1115, the code image composition unit 106 updates the value of the pixel counter by subtracting the skip length value from the current value of the pixel counter.

  Next, in step S1116, the code image composition unit 106 determines whether or not the value of the pixel counter is 0 (zero). If the result of this determination is that the value of the pixel counter is 0 (zero), in steps S1117 and S1118, the code image composition unit 106 performs the same processing as in steps S1106 and S1107, and returns to step S1102. That is, the code image synthesis unit 106 reads one entry of the code image data of the first frame Frame1. Then, the code image composition unit 106 sets the run length of the read code image data entry as the pixel counter value, and sets the pixel value of the read code image data entry as the Frame1 output pixel value. . On the other hand, if the value of the pixel counter is not 0 (zero), steps S1117 and S1118 are omitted and the process returns to step S1102.

  If it is determined in step S1114 that the value of the pixel counter is not greater than or equal to the skip length, the skip length is longer than the number of remaining pixels in the entry of the encoded image data of the first frame Frame1 being processed. In this case, the code image data entry of the new first frame Frame1 is read in the processing of steps S1119 to S1122, and the code image data entry of the first frame Frame1 corresponding to the skip length is skipped.

For this purpose, first, in step S1119, the code image composition unit 106 subtracts the value of the pixel counter from the skip length value. In steps S1120 and S1121, the code image composition unit 106 performs the same processing as steps S1106 and S1107.
In step S1122, the code image composition unit 106 determines whether the skip length value is 0 (zero). If the result of this determination is that the skip length value is 0 (zero), processing returns to step 1102. On the other hand, when the skip length value is not 0 (zero) (a positive value), the processes in steps S1114 to S1122 are repeated until the skip length value becomes 0 (zero).

Hereinafter, an example of a method in which the code image synthesis unit 106 processes the entry 1001 to the entry 1003 of the area determination data illustrated in FIG. 10 will be described.
The code image synthesizing unit 106 sequentially executes the processes of steps S1101 to S1103, and reads the top entry 1001 of the area determination data illustrated in FIG.
Since the source value of the entry 1001 in FIG. 10 is Frame1, the process proceeds to step S1105 as a result of the determination in step S1104.
In the initialization process in step S1101, the value of the pixel counter is reset to 0 (zero). Therefore, the process proceeds to step S1106 as a result of the determination in step S1105. In step S1106, the code image composition unit 106 reads out the head entry 910 in FIG. 9B as the code image data entry of the first frame Frame1.

Subsequently, in step S1107, the code image composition unit 106 sets the run length 12 of the entry 910 in FIG. 9B as the value of the pixel counter. Further, the code image synthesis unit 106 sets the pixel value Color0 of the entry 910 in FIG. 9B as the Frame1 output pixel value.
In step S1103, the entry 1001 (run length 2) in FIG. 10 is read. In step S1107, the pixel value Color0 of the entry 910 is set as the Frame1 output pixel value. Therefore, in step S1108, the code image composition unit 106 outputs an entry with the run length of 2 and the pixel value of Color0 as the entry 1201 of the code image data of the second frame Frame2.
In step S1107, the value of the pixel counter is updated to 12. In step S1103, the entry 1001 (run length 2) in FIG. 10 is read. Therefore, in step S1109, the code image composition unit 106 subtracts 2 which is the length of the entry 1001 from the current value 12 of the pixel counter, and updates the value of the pixel counter to 10.

Thereafter, the processes of steps S1102 and S1103 are executed, and the code image composition unit 106 reads the next entry 1002 of the area determination data shown in FIG.
Since the source of the entry 1002 is Frame 2, the process proceeds to step S1110 as a result of the determination in step S1104.
In step S1110, the code image synthesis unit 106 reads the top entry 801 in FIG. 8 as an entry of the partial code image data of the second frame Frame2. Subsequently, in step S <b> 1111, the code image composition unit 106 sets the pixel value Color <b> 7 of the entry 801 read out in step S <b> 1110 as the Frame2 output pixel value.

Next, in step S1112, the code image synthesis unit 106 outputs an entry 1202 of code image data of the second frame Frame2. This entry 1202 is an entry having the Frame2 output pixel value Color7 set in step S1110 as the pixel value and the length 1 of the entry 1002 of the area determination data read out in step S1103 as the run length.
Next, in step S1113, the code image composition unit 106 sets the length value 1 of the entry 1002 of the area determination data read in step S1103 as a skip length.

In step S1109, the current value of the pixel counter is set to 10. In step S1113, the skip length value is set to 1. As a result of the determination in step S1114, the process proceeds to step S1115.
In step S1115, the code image composition unit 106 subtracts 1 as the skip length value from 10 as the current value of the pixel counter, and updates the value of the pixel counter to 9. As a result, in step S1116, since it is determined that the pixel counter is not 0, the process returns to step S1102.

Then, the processes of steps S1102 and S1103 are executed, and the code image composition unit 106 reads the next entry 1003 of the area determination data shown in FIG.
Since the source of the entry 1003 is Frame1, the process proceeds to step S1105 as a result of the determination in step S1104.
In step S1115, the value of the pixel counter is updated to 9. Therefore, as a result of the determination in step S1105, the process proceeds to step 1108. In the previous step S1107, the pixel value Color0 of the entry 910 is set as the Frame1 output pixel value. Therefore, in step S1108, the code image composition unit 106 outputs an entry with the run length of 9 and the pixel value of Color0 as the entry 1203 of the code image data of the second frame Frame2.

As described above, in step S1115, the value of the pixel counter is updated to 9. In step S1103, the entry 1003 (run length 9) in FIG. 10 is read. Therefore, in step S1109, the code image composition unit 106 subtracts 9 which is the length of the entry 1003 from the current value 9 of the pixel counter, and updates the value of the pixel counter to 0.
Thereafter, by performing the same processing, the code image data of the second frame Frame2 shown in FIG. 12 can be obtained.

Returning to the description of FIG. 1, the output unit 107 outputs the code image data of the second frame Frame 2 generated by the code image synthesis unit 106 as described above to the data storage unit 108.
When the output unit 107 completes outputting one frame (the encoded image data of the second frame Frame2), the decoding unit 109 reads the decoded frame from the data storage unit 108 and decodes it.
The display unit 110 displays the data output from the decoding unit 109.
In addition, in a system that superimposes and outputs a plurality of images other than the image drawn by the drawing unit 104, such as a video and a photograph, a processing unit that superimposes images is added between the decoding unit 109 and the display unit 110. May be.

  As described above, in the present embodiment, an area to be updated with respect to the area of the previous frame is specified from the drawing command. Then, from the encoded image data of the previous frame, the data of the specified area is read and decoded, and the partial encoded image data encoded by overwriting the decoded data with the updated area data of the current frame Generate. Then, the code image data of the previous frame and the partial code image data are merged with reference to the area updated with respect to the area of the frame of the previous frame. In this way, since the encoded image data of the previous frame is copied and the updated area is decoded and drawn, the drawing time can be shortened compared to the method of redrawing all objects. Further, since the encoded image data and the partial encoded image data of the previous frame are encoded, the bandwidth necessary for outputting the drawn data can be reduced as compared with the case where uncompressed data is used.

(Modification)
In the present embodiment, the banding box is calculated even for a transparent object, and is included in the update area information. However, such processing may not be performed for transparent objects. Hereinafter, an example of a method for calculating the bounding box and generating the update area information in this case will be described.

  The update area extraction unit 102 holds the α value of the drawing command related to the drawing color instruction, and does not perform the calculation of the bounding box of the object drawn with α = 0 indicating transparency. In the drawing command shown in FIG. 2, commands 202, 205, 209, and 213 are related to the drawing color instruction. In the drawing command of FIG. 2, the α value becomes 0 after the α value is set to 0 by the command 205 and before the command 207 is executed. For this reason, the update area extraction unit 102 does not generate a bounding box for an object to be drawn based on the command 206 using the color set by the command 205.

FIG. 13 is a diagram illustrating another example of the update area information. Specifically, FIG. 13A is a diagram illustrating an example of an update region in which the bounding boxes 402 to 404 illustrated in FIG. 4 are mapped to a two-dimensional coordinate system with pixel accuracy of 8 pixels in length and 10 pixels in width. FIG. 13B is a diagram showing an example of update area information generated from the update area shown in FIG.
As shown in FIG. 13A, only the areas 502, 503, and 504 of the object drawn with the set colors of the commands 202, 209, and 213 are update areas. As illustrated in FIG. 13B, the update area information includes three entries 1304, 1305, and 1306. The update area information is output from the update area extraction unit 102 to the partial image decoding unit 103 and the code image synthesis unit 106.

Further, in the present embodiment, the case where the calculation of the bounding box is performed using the same pixel accuracy as the drawing has been described as an example. However, in order to speed up the calculation, the bounding box may be calculated with a pixel accuracy (unit of a plurality of pixels) lower than that of drawing.
FIG. 14 is a diagram illustrating another example of the update area information. Specifically, FIG. 14A is a two-dimensional representation of the bounding boxes 402 to 404 shown in FIG. 4 with pixel accuracy of 4 pixels in the vertical direction and 5 pixels in the horizontal direction (pixel accuracy in the X direction and Y direction are each half of the drawing accuracy). It is a figure which shows an example of the update area mapped on the coordinate system. FIG. 13B is a diagram showing an example of update area information generated from the update area shown in FIG.
As shown in FIG. 14A, the areas 1401 and 1402 have the same size as the areas 501 and 502 in FIG. On the other hand, since the calculation accuracy of the bounding box is halved, the areas 1403 and 1404 are twice and four times larger than the areas 503 and 504 in FIG. 5A, respectively. Accordingly, the size of the update area information entry 1410 in FIG. 14B is twice as large as the size of the update area information entry 510 in FIG. 5B. Also, the size of the update area information entry 1411 in FIG. 14B is twice as large as the size of the update area information entry 511 in FIG. 5B.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

(Other examples)
The present invention is also realized by executing the following processing. That is, first, software (computer program) for realizing the functions of the above embodiments is supplied to a system or apparatus via a network or various storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads and executes the computer program.

  102 update region extraction unit, 103 partial image decoding unit, 106 code image synthesis unit

Claims (11)

Setting means for setting an update area to be updated by the second image among the areas of the first image based on a drawing instruction of the second image input after the first image;
The encoded image data of the first image in the update area is decoded from the encoded image data of the first image stored in the storage medium, and decoded data in the update area of the first image is generated. Decryption means;
A drawing process for overwriting the second image is performed on the decoded data in the update area of the first image generated by the decoding unit, and the area including the update area of the second image is drawn. Drawing means for generating data and outputting drawing data of the update area of the second image;
Encoding means for encoding drawing data in the update area of the second image output by the drawing means;
Coded image data in an area different from the update area of the first image, and drawing data in the update area of the second image encoded by the encoding means, stored in the storage medium Code image synthesizing means for generating code image data of the second image,
Storage means for storing code image data of the second image generated by the code image synthesis means in the storage medium;
An image processing apparatus comprising:   The setting means outputs update area information including identification information for uniquely identifying an object included in the update area, and information for specifying a position and a size of the update area. The image processing apparatus according to 1.   The setting means refers to the color information of the object included in the drawing instruction of the second image, and does not set the transparent object area as the update area, but sets the non-transparent object area as the update area. The image processing apparatus according to claim 1, wherein the image processing apparatus is set.   The image processing apparatus according to claim 1, wherein the setting unit sets a set of rectangular areas in units of pixels set in advance as the update area.   The setting means sets, as the update area, a rectangle having the same pixel accuracy as the pixel precision of drawing by the drawing means, or a set of rectangular areas having a pixel accuracy lower than the pixel precision of drawing by the drawing means. The image processing apparatus according to claim 4. A setting step of setting an update area to be updated by the second image among the areas of the first image based on a drawing instruction of the second image input after the first image;
The encoded image data of the first image in the update area is decoded from the encoded image data of the first image stored in the storage medium, and decoded data in the update area of the first image is generated. Decryption step;
A drawing process for overwriting the second image is performed on the decoded data in the update area of the first image generated by the decoding step, and the area including the update area of the second image is drawn. A drawing step of generating data and outputting drawing data of the update area of the second image;
An encoding step of encoding drawing data in the update region of the second image output by the drawing step;
Coded image data in a region different from the update region of the first image stored in the storage medium, drawing data in the update region of the second image encoded by the encoding step, and A code image synthesis step for generating code image data of the second image,
A storage step of storing the code image data of the second image generated by the code image synthesis step in the storage medium;
An image processing method comprising:   The setting step outputs update area information including identification information for uniquely identifying an object included in the update area and information for specifying a position and a size of the update area. 6. The image processing method according to 6.   The setting step refers to the color information of the object included in the drawing instruction for the second image, and does not set the transparent object area as the update area, but sets the non-transparent object area as the update area. The image processing method according to claim 6, wherein the image processing method is set.   The image processing method according to claim 6, wherein the setting step sets a set of rectangular areas in units of pixels set in advance as the update area.   The setting step includes setting, as the update region, a rectangle having the same pixel accuracy as the pixel accuracy of drawing by the drawing step or a set of rectangular regions having a pixel accuracy lower than the pixel accuracy of drawing by the drawing step. The image processing method according to claim 9, wherein:   A computer program for causing a computer to execute each step of the image processing method according to claim 6.

Images