JP2015210375A - Image processing apparatus, control method thereof, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve display speed of graphic data.SOLUTION: An image processing apparatus which allocates sub-image data formed by dividing image data constituting a video image, to be processed by different image processing sections in parallel includes: extraction means which extracts graphic data that moves with time, out of graphic data to be synthesized with the image data; dividing means which divides the graphic data at a position crossing a boundary between adjacent frames when the graphic data extracted by the extraction means crosses the boundary of the frames due to displacement; and determination means which determines the order of generating multiple pieces of sub-graphic data divided by the dividing means, on the basis of displacement information of the graphic data, for graphic generation means of the image processing section.

Description

  The present invention relates to a technique for generating graphic data to be combined with divided image data obtained by dividing image data by a plurality of graphic generation means.

  Currently, Full HD (1920 × 1080) is a common resolution for digital television. However, standardization of higher-resolution image data such as 4K2K (4096 × 2160) and Super Hi-Vision (7680 × 4320) has progressed, and devices capable of displaying such high-resolution image data have begun to be provided.

  High-resolution image data reception processing, image processing, and output processing to a display device require a wider-band image input / output interface and memory interface, and the amount of computation increases. It may not be possible to process. Therefore, in order to perform reception processing, image processing, and output processing of high resolution image data, a method of dividing high resolution image data in space and performing image processing in units of divided image data is often used. For example, high resolution image data may be divided into left and right halves, and image processing of each divided image data may be performed in parallel by two image processing LSI chips.

  Further, the image processing LSI chip processes not only image data. That is, the image processing LSI chip also has a function of generating graphic data such as information related to displayed image data, a setting menu, a Web screen, and a photograph, and superimposing them on the image data for output. As the resolution of the image data is increased, the graphic data to be superimposed is also increased in resolution, and the amount of calculation in the graphic data generation process increases. Therefore, similarly to the image processing of the high resolution image data, the graphic data is generated using the graphic generation unit provided in the plurality of image processing LSI chips.

  The graphic data that the image processing LSI chip superimposes on the image data has a different display area, unlike the image data. For example, the graphic data may be displayed on the entire surface of the image data, or may be displayed only on the right half of the image data. Further, the load in the graphic data generation process is not constant for the entire graphic data. For example, if graphic data includes a region with a complex image such as a photograph and a region containing only characters, the load on the graphic data generation processing for the region containing the image is greater than that of a region containing only characters. . Therefore, in order to generate graphic data at high speed, it is necessary to perform processing so that the processing load is uniform among a plurality of graphic generation units that generate graphic data. Then, it is necessary to make maximum use of the graphic data generation processing capability provided in the image processing system composed of a plurality of image processing LSI chips.

  In Patent Document 1, the processing load related to image formation of an image block obtained by dividing graphic data into rectangles is estimated, and the processing is distributed to a plurality of processing units so that the processing load is uniform, thereby speeding up the distributed processing. A technique for achieving the above is disclosed.

JP 2010-162745 A

  However, the technique disclosed in Patent Document 1 requires processing for estimating the processing load of graphic data. Furthermore, in this technique, not only the estimation result but also the graphic data display area is taken into consideration, and the graphic data generation process is assigned. For this reason, there is a problem that the processing load for determining the allocation of the graphic data generation processing is increased, and the display speed of the graphic data is reduced.

  Further, in this technique, generation of graphic data divided into rectangles using spatial information at a certain point in time, that is, information indicating a responsible area in a space allocated to each image processing chip and information indicating a graphic display position is generated. The assignment will be decided. Therefore, in this technique, when considering time-axis information, that is, graphic data before and after the passage of time, useless graphic data may be generated. More specifically, the same graphic data may be generated more than necessary in an image processing system including a plurality of image processing chips. In an image processing system including a plurality of image processing chips, memory transfer of graphic data may occur more than necessary between image processing chips instead of generating graphic data. Therefore, there is a problem that the graphic generation processing capability provided in the image processing system including a plurality of image processing chips cannot be used efficiently. As a result, the display speed of graphic data is reduced.

  For example, consider graphic data that translates the boundary between an assigned area assigned to one image processing chip after the space is divided and an assigned area assigned to another image processing chip. The graphic data that moves in parallel with the passage of time changes only at the display position in the next frame. That is, if each image processing chip once holds the generated graphic data, it is necessary to generate the graphic data again even if it is necessary to display the graphic data in the next frame, for example. Absent. However, when an image processing chip that has finished generating graphic data in its own area generates graphic data in the area in charge of another image processing chip, the graphic that intrudes into its area in the next frame Assume that graphic data other than data is generated. In that case, the image processing chip that has finished generating graphic data in its own area needs to generate graphic data that enters the area in its own area in the next frame. In addition, even if the image processing chip generates graphic data that enters its own area in the next frame, when the generated graphic data is transferred to the memory of another image processing chip, If deleted, the same graphic data must be generated again in the next frame.

  Therefore, an object of the present invention is to efficiently generate graphic data and improve the display speed of graphic data.

  Therefore, the present invention is an image processing apparatus that assigns the processing of each divided image data obtained by dividing the image data constituting the moving image to different image processing units and processes them in parallel, and the graphic data to be combined with the image data Extraction means for extracting graphic data that moves with the passage of time, and when the graphic data extracted by the extraction means moves across the boundary of the divided image data in the previous and subsequent frames, the graphic data is at least in the previous and next frames And dividing means for dividing at a position intersecting with the boundary, and the generation order of the plurality of divided graphic data divided by the dividing means based on the movement information of the graphic data to the graphic generating means of the image processing unit. And determining means for determining.

  According to the present invention, it is possible to efficiently generate graphic data and improve the display speed of graphic data.

It is a figure which shows an example of a structure of an image processing system. FIG. 3 is a diagram (part 1) illustrating an example of division of graphic data. FIG. 6 is a second diagram illustrating an example of division of graphic data. FIG. 10 is a third diagram illustrating an example of division of graphic data. It is a figure which shows an example of the management table which a production | generation graphic management part manages.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.
<Embodiment 1>
FIG. 1 is a diagram illustrating an example of a configuration of an image processing system according to the present embodiment. The image processing system in the present embodiment divides high-resolution image data constituting a high-resolution video into two image data (divided image data), and image processing of each divided image data is performed in parallel by two image processing chips. To do. The image processing system shown in FIG. 1 is an example of an image processing apparatus that processes an image.

  In FIG. 1, an image processing chip A101 has a function of executing image processing and graphic generation processing, superimposing graphic data on image data, and then outputting the graphic data to a display device. As shown in FIG. 1, the image processing chip A 101 includes a receiving unit 102 that receives image data, an image processing unit 103 that performs predetermined processing on the image data, a processor 107 that controls graphic data generation processing, and graphic data generation. A graphic generation unit 106 that performs processing, a synthesis unit 104 that superimposes graphic data on image data, and an output unit 105 that outputs image data to a display device are provided. The memory 108 is a memory (storage area) corresponding to the image processing chip A101. Further, the processor 107 includes a moving graphic extraction unit 109, a graphic division unit 110, an order determination unit 111, and a holding command addition unit 112. In addition, the graphic generation unit 106 includes a generated graphic management unit 113.

  The image processing system in this embodiment includes two chips, an image processing chip A101 and an image processing chip B121 equivalent to the image processing chip A101, in order to process high-resolution image data. The image processing chip A101 is in charge of the display area of the left half of the display device. That is, the image processing chip A 101 performs image processing on image data (hereinafter referred to as left image data) 130 displayed in the left half display area of the display device, and the left image data 131 after the image processing and the graphics are processed. After combining the data, it is output to the display device. More specifically, when receiving the left image data 130, the receiving unit 102 outputs the left image data 130 to the image processing unit 103. The image processing unit 103 performs image processing such as IP (interlace / progressive) conversion and edge enhancement on the left image data 130, and outputs the left image data 131 after the image processing to the combining unit 104. The combining unit 104 reads the graphic data from the memory 108 and superimposes it on the left image data 131 after the image processing, thereby generating left display image data 139. Then, the combining unit 104 outputs the left display image data 139 to the output unit 105. The output unit 105 displays the left display image data 139 by outputting the left display image data 139 to the display device. In FIG. 1, areas C, D, and E in the left display image data 139 are graphic data synthesized so as to be superimposed on the left image data 131 after image processing.

  Similarly, the image processing chip B121 is responsible for the display area of the right half of the display device. In other words, the image processing chip B121 performs image processing on image data (hereinafter referred to as right image data) 132 displayed in the right half display area of the display device. Then, the image processing chip B121 synthesizes the right image data 133 after the image processing and the graphic data, and then outputs them to the display device as the right display image data 140. In FIG. 1, a region F in the right display image data 140 is graphic data synthesized so as to be superimposed on the right image data 133 after image processing.

  The memory 124 is a memory (storage area) corresponding to the image processing chip B121. That is, the graphic data combined with the right image data 133 after image processing is graphic data read from the memory 124. The receiving unit 102, the image processing unit 103, the synthesizing unit 104, and the output unit 105 in the image processing chip B121 are the same as the receiving unit 102, the image processing unit 103, the synthesizing unit 104, and the output unit 105 in the image processing chip A101, respectively. It is a configuration. The graphic generation unit 122 includes a generated graphic management unit 125.

  The display device displays the left display image data 139 and the right display image data 140 together as one high resolution image data. In the present embodiment, for the sake of convenience of explanation, the memories 108 and 124 are memories for storing graphic data. However, in reality, the image processing units 103 of the image processing chip A 101 and the image processing chip B 121 have moving image data. It is also used as a frame memory for storing each image data constituting the.

  A user instruction 151 input via a remote controller or a keyboard is input to the processor 107 of the image processing chip A101. Examples of the graphic data include information on displayed image data, a setting menu, a Web screen, a photograph, and the like, and display sizes and display positions are various. The graphic data may be displayed across the boundary between both image areas obtained by dividing the high resolution image data, or may be displayed only on one image area. Further, the graphic data may be displayed by being moved to a position where it can be easily viewed by a user using a remote controller or the like, or may be displayed by being translated.

  The processor 107 determines graphic data generated from the user instruction 151 and converts it into a graphic generation command that can be interpreted by the graphic generation units 106 and 122. An example of the graphic generation instruction is a run-length instruction that replaces a sequence of consecutive identical symbols with a number indicating the length of the symbol sequence. Depending on the characteristics of the graphic data, for example, whether it is a graphic, a character, or a photograph, it is also efficient to convert it into a graphic generation command while switching to an appropriate command format for each part. Generally used for generating graphic data. Also known is a method of dividing graphic data into several rectangles and generating a graphic generation command for each division unit. In the following description, each divided graphic data is referred to as divided graphic data.

  When receiving the graphic generation command, the graphic generation units 106 and 122 generate graphic data such as bitmap data. The graphic generation units 106 and 122 may be configured with a processor and firmware in order to flexibly cope with a change in the specification of the graphic generation instruction, or may be configured with full hardware, with high-speed graphic data generation as a top priority. There is also a case.

  The processor 107 of the image processing chip A 101 controls graphic data generation processing of the entire image processing system. In other words, the processor 107 of the image processing chip A101 generates graphic data by controlling the graphic generation unit 106 of the image processing chip A101 and the graphic generation unit 122 of the image processing chip B121. Note that the processor 123 of the image processing chip B121 is stopped because it is not used in this embodiment.

  Each of the graphic generation units 106 and 122 temporarily stores the generated graphic data in a memory corresponding to the image processing chip to which the graphic generation unit 106 and 122 belong. After that, the graphic generation units 106 and 122 respectively generate the generated graphic data when the image data area to be combined with the generated graphic data is an area handled by another image processing chip to which the graphic data does not belong. Transfer to the memory corresponding to the image processing chip. For example, the graphic generation unit 122 of the image processing chip B 121 generates divided graphic data 136 and 137 for the region F and the region E and stores them in the memory 124. Here, the divided graphic data 137 of the region E is divided graphic data combined with the left image data 131 after the image processing. Therefore, the divided graphic data 137 of the area E is transferred as the divided graphic data 138 of the area E from the memory 124 of the image processing chip B 121 to the memory 108 of the image processing chip A 101.

  For communication between the processor 107 of the image processing chip A 101 and the graphic generation unit 122 of the image processing chip B 121, a general communication route when parallel processing is performed by a plurality of chips, such as a PCI bus or serial communication, is used. Similarly, the image processing chip A 101 and the image processing chip B 121 are generally connected by a high-speed data transfer bus such as PCI Express in order to refer to data on the other side in image processing or the like. Can be accessed. In other words, the image processing chip A101 can access the memory 124 corresponding to the image processing chip B121. Similarly, the image processing chip B121 can access the memory 108 corresponding to the image processing chip A101. The divided graphic data 138 in the area E is transferred from the memory 124 to the memory 108 using the above route.

  The moving graphic extracting unit 109, the graphic dividing unit 110, the order determining unit 111, and the holding instruction adding unit 112 may be implemented as hardware in the processor 107, or read out from an HD (Hard Disk) (not shown) by the processor 107. You may implement as software implement | achieved by running a program.

  The moving graphic extraction unit 109 extracts graphic data that moves in parallel with the passage of time from graphic data that the processor 107 determines to display based on the user instruction 151. Then, the moving graphic extracting unit 109 notifies the graphic dividing unit 110 and the order determining unit 111 of information on the graphic data to be translated. More specifically, the processor 107 once represents graphic data to be displayed in a vector image language such as SVG (Scalable Vector Graphics). In SVG, parallel moving graphic data is expressed separately as graphic information and movement information. The movement information includes movement amount information and movement direction information, which will be described later. Accordingly, the moving graphic extraction unit 109 can extract parallel moving graphic data from the vector image information generated by the processor 107. Further, the moving graphic extraction unit 109 can extract the moving amount information and moving direction information per unit time of the graphic data moving in parallel from the vector image information. Thereby, the moving graphic extraction unit 109 can specify the movement of the graphic data that translates on the display image data, and can obtain the position corresponding to the time of the graphic data that translates.

  The graphic dividing unit 110 divides the generated graphic data into a plurality of rectangular areas which are units generated by the graphic generating units 106 and 122. When dividing the graphic data, the graphic dividing unit 110 divides the graphic data at least at the boundary when the graphic data crosses the boundary of the divided image data. Further, the graphic dividing unit 110, when the parallel moving graphic data extracted by the moving graphic extracting unit 109 moves across the boundaries of the divided image data, the graphic data is at least in the temporally preceding and following frames. It divides | segments in the position which crosses, ie, the position which cross | intersects.

  An example of division of graphic data will be described with reference to FIG. The display image data 201 corresponds to the left display image data 139 in FIG. The display image data 202 corresponds to the right display image data 140 in FIG. In FIG. 2, the divided graphic data divided into regions C, D, E, and F is superimposed on the image data. Here, the character information such as “C”, “D”, “E”, “F” is a unique ID assigned to each of the divided graphic data. When the graphic data is the same at different times, the assigned ID is the same. On the other hand, if the graphic data is different, a new ID is assigned. Note that although the ID is character information here, it is not necessary to be limited to this. For example, numbers such as “1”, “2”, and “3” may be used.

  In FIG. 2, display image data 201 and 202 are display image data at time T, which is the current frame, for example. The display image data 203 and 204 are display image data at time T + 1 of the next frame. FIG. 2 shows that the graphic data is translated in the right direction between the previous and subsequent frames. The graphic dividing unit 110 divides the graphic data area E and area F at least at the boundary between the display image data 201 and 202 at time T. Further, the graphic dividing unit 110 is an area where the graphic data area E crosses the boundary between the display image data 203 and 204 from the time T to the time T + 1 from the movement amount information and movement direction information of the graphic data. Is specified. The graphic dividing unit 110 divides the area D and area E of the graphic data at least at the boundary between the display image data 203 and 204. That is, when the graphic data is divided into regions C, D, E, and F, the region E and the region F are divided based on the boundary between the display image data 201 and 202, and the region D and the region E are displayed. The image data 203 and 204 are divided based on the boundary. Therefore, the edge of the divided graphic data of the area E comes to the boundary of the area division of the display image data 201 at the time T and the boundary of the area division of the display image data 204 at the time T + 1.

  The order determination unit 111 determines in which order the graphic generation unit of each image processing chip generates the divided graphic data of a plurality of areas divided by the graphic division unit 110. The order determination unit 111 determines the generation order of the divided graphic data by the graphic generation unit according to the following rules. Here, for convenience of explanation, a case where the graphic generation unit C belonging to the image processing chip C (not shown) generates divided graphic data will be described as an example.

Rule 1: The graphic generation unit C generates divided graphic data in the assigned image area of the image processing chip C to which the graphic generation unit C belongs. At that time, the order determination unit 111 determines the generation order so that the graphic generation unit C generates the divided graphic data in the order from the image boundary of the image area in charge of the image processing chip C to the region division boundary. The image boundary here means the left end in the case of the display image data 201 in FIG. 2 and the right end in the case of the display image data 202. Further, the region division boundary means a boundary between the display image data 201 and 202 in FIG.
Rule 2: When the graphic generation unit C generates the divided graphic data in the image area in charge of another image processing chip different from the image processing chip C, the order determination unit 111 will be responsible for the image area in charge of the image processing chip C in the future. The generation order is determined so as to generate with priority from the divided graphic data that is translated in parallel.
Rule 3: The order determination unit 111, when the graphic generation unit C generates divided graphic data in the image area in charge of the image processing chip other than the above, the same direction as that of the rule 1, that is, the image area in charge of the graphic generation unit C The generation order is determined so that the divided graphic data is generated in the order of the direction from the image boundary to the region dividing boundary.
Further, the order determining unit 111 notifies the holding instruction adding unit 112 of the information of the divided graphic data whose order is determined by the rule 2.

  The rules described above will be described more specifically with reference to FIGS. As described above, the display image data 201 and 202 in FIG. 2 indicate display image data at the time T, and the display image data 203 and 204 indicate display image data at the time T + 1. The graphic data divided into the regions C, D, E, and F is graphic data that translates in the right direction. An arrow 2011 on the display image data 201 and an arrow 2021 on the display image data 202 indicate the directions from the image boundary of each image region to the region division boundary. The order determining unit 111 applies the divided graphic data of the regions 1 to C, D and E from rule 1 to the region F from rule 3 to the graphic generation unit belonging to the image processing chip in charge of the image region of the display image data 201. Determine the generation order. That is, the order determination unit 111 determines the generation order so that the graphic generation unit generates divided graphic data in the order of regions C, D, E, and F.

  Further, the order determination unit 111 applies the rule 1 to the region F, the rule 2 to the region E, and the rule 3 to the region D, for the graphic generation unit belonging to the image processing chip in charge of the image region of the display image data 202. The generation order of the divided graphic data C is determined. That is, the order determination unit 111 determines the generation order so that the graphic generation unit generates the divided graphic data in the order of the regions F, E, D, and C. Furthermore, since the order determination unit 111 determines the generation order of the region E by applying the rule 2, the order determination unit 111 notifies the holding instruction addition unit 112 of the information.

  In FIG. 3, display image data 301 and 302 indicate, for example, display image data at time T that is the current frame, and display image data 303 and 304 indicate display image data at time T + 1 of the next frame. Yes. The graphic data divided into the areas E, F, G, and H is graphic data that translates in the right direction. The graphic data divided into the areas C and D and the graphic data divided into the areas I and J are superimposed on the same display area at time T and time T + 1, but are different graphic data. An arrow 3011 on the display image data 301 and an arrow 3021 on the display image data 302 indicate the directions from the image boundary of each image region to the region division boundary. The order determination unit 111 determines the generation order of the divided graphic data of the areas E, F, C, and G from the rule 1 for the graphic generation unit belonging to the image processing chip in charge of the image area of the display image data 301. When applying the rule 3, the order determination unit 111 may set the generation order of the divided graphic data in either of the regions D and H first. Here, the order determination unit 111 determines the generation order so that the graphic generation unit generates divided graphic data in the order of regions E, F, C, G, D, and H.

  Further, the order determining unit 111 applies the rules 1 to H, D, the rule 2 to the region G, and the rule 3 to the region with respect to the graphic generation unit belonging to the image processing chip in charge of the image region of the display image data 302. The generation order of divided graphic data C, F, and E is determined. That is, the order determination unit 111 determines the generation order so that the graphic generation unit generates divided graphic data in the order of regions H, D, G, C, F, and E. Furthermore, since the order determination unit 111 determines the generation order of the region G by applying the rule 2, the order determination unit 111 notifies the holding instruction addition unit 112 of the information.

  The processor 107 converts a vector image language such as SVG into a graphic generation command in units of divided graphic data, that is, for each divided graphic data. Then, the processor 107 notifies the graphic generation units 106 and 122 to generate the divided graphic data in the generation order determined by the order determination unit 111. More specifically, the processor 107 notifies the graphic generation unit 106 as the generation command 152 in the order in which the generation commands for each divided graphic data are generated. Similarly, the processor 107 notifies the graphic generation unit 122 as a generation instruction 153 in the order in which generation instructions for each divided graphic data are generated. The generation command described above includes the ID assigned to the divided graphic data and the coordinate information on the display image data displayed by superimposing the divided graphic data. Furthermore, the holding instruction adding unit 112 generates a divided graphic data generation instruction whose generation order is determined by the above-described rule 2 based on the generation order information of the divided graphic data obtained by the notification from the order determining unit 111. A retention command for retaining the divided graphic data is added. For example, in the case of FIG. 2, the retention command adding unit 112 responds to the region E divided graphic data generation command 153 to the graphic generation unit 122 of the image processing chip B 121 that is responsible for the image region of the display image data 202. When the divided graphic data is generated, a holding instruction to be held is added. As a result, even if the divided graphic data once generated is not necessary again, it is not necessary to generate it again, so the generation efficiency of graphic data can be improved, and as a result, the display speed of graphic data can be improved. Can do.

  When the graphic generation units 106 and 122 receive a generation command for each divided graphic data, the graphic generation units 106 and 122 refer to a management table managed by the generated graphic management units 113 and 125, which will be described later with reference to FIG. Generate data. The generated graphic management units 113 and 125 update the graphic data and the management table stored in the memory of the image processing chip to which they belong.

  A management table managed by the above-described generated graphic management units 113 and 125 will be described with reference to FIG. FIG. 5 is a diagram illustrating an example of a management table managed by the generated graphic management units 113 and 125. In the management table shown in FIG. 5, information on generated divided graphic data existing in the memory of the image processing chip to which each of the generated graphic management units 113 and 125 belongs is registered. That is, the generated graphic management unit 113 manages the information of the generated divided graphic data existing in the memory 108 of the image processing chip A 101 in its own management table. Further, the generated graphic management unit 125 manages the information of the generated divided graphic data existing in the memory 124 of the image processing chip B 121 in its own management table. The generated graphic management units 113 and 125 manage the information of the generated divided graphic data in the management table by using the ID assigned to the divided graphic data sent included in the generation command. In addition, the generated graphic management units 113 and 125 register the memory address, data size information, and the like where the generated divided graphic data is stored in the management table as accompanying information of the generated divided graphic data.

  When the graphic generation units 106 and 122 receive a generation command of the divided graphic data to be superimposed on the image area handled by the image processing chip to which the graphic generation unit 106 belongs, the graphic generation units 106 and 122 refer to the management table based on the ID included in the generation command. To do. Then, if the ID that matches the management table is registered, the graphic generation units 106 and 122 do not generate the divided graphic data in the received generation instruction, and send generation completion notifications 154 and 155 to the processor 107. On the other hand, if the ID that matches the management table is not registered, the graphic generation units 106 and 122 generate the divided graphic data in the received generation instruction and store it in the memory of the image processing chip to which the graphic generation unit 106 and 122 belong. Generation completion notifications 154 and 155 are sent to the processor 107. At this time, the generated graphic management units 113 and 125 update the management table managed by themselves. More specifically, the generated graphic management units 113 and 125 add information on the generated divided graphic data to the management table and update it.

  When the graphic generation units 106 and 122 receive a generation command of divided graphic data to be superimposed on an image area other than the image area handled by the image processing chip to which the graphic generation unit 106 belongs, from the processor 107, the graphic generation unit 106 and 122 Generate data. The graphic generation units 106 and 122 temporarily store the generated divided graphic data in the memory of the image processing chip to which the graphic generation unit 106 and 122 belong, and transfer the divided graphic data to the memory of the image processing chip in charge of the other image area. Then, the graphic generation units 106 and 122 send generation completion notifications 154 and 155 to the processor 107. Thereafter, the generated graphic management units 113 and 125 delete the generated divided graphic data from the memory of the image processing chip to which the generated graphic management units 113 and 125 have a limited memory capacity. However, the generated graphic management units 113 and 125, when the retention command by the retention command adding unit 112 is added to the generated divided graphic data generation command, the generated divided graphic data will belong to the image processing to which it belongs in the future. Since the chip is used in an image area for which the chip is in charge, the generated divided graphic data information is added to the management table and updated without being deleted from the memory. In addition, the generated graphic management units 113 and 125 may delete the divided graphic data that is no longer used and exists in the memory of the image processing chip to which the generated graphic management unit 113 and 125 belong, or delete information registered in the management table. Here, the generated graphic management units 113 and 125 may specify the divided graphic data that is not used based on the user's instruction, or the divided graphic data that has been stored in the memory for a predetermined time or longer is not used. You may make it identify with a thing. In this way, the generated graphic management units 113 and 125 can use the memory of the image processing chip more effectively.

  The series of processes described above will be described more specifically with reference to FIGS. Here, the case of time T in FIG. 2, that is, the case where the display image data 201 and 202 are displayed will be described as an example. When the graphic generation unit 106 generates the divided graphic data 134 of the area C and stores it in the memory 108, it sends a generation completion notification 154 to the processor 107. Next, when the graphic generation unit 106 generates the divided graphic data 135 of the region D and stores it in the memory 108, the graphic generation unit 106 sends a generation completion notification 154 to the 107 processor. Thereafter, the graphic generation unit 106 starts generating divided graphic data for the next region E.

  When the graphic generation unit 122 generates the divided graphic data 136 for the region F and stores it in the memory 124, it sends a generation completion notification 155 to the processor 107. At this stage, when the generation of all the divided graphic data has not been completed yet, the graphic generation unit 122 generates the divided graphic data of the assigned image area of another image processing chip different from the image processing chip B121 to which the graphic generation unit 122 belongs. Start generation. That is, in the case of FIG. 2, the graphic generation unit 122 starts generating the divided graphic data 137 of the area E in the assigned image area of the image processing chip A101. Since the divided graphic data 137 of the area E is superimposed on the assigned image area of the image processing chip A 101, the graphic generation unit 122 generates the divided graphic data 137 of the area E, stores it in the memory 124, and then stores it in the memory 108. The divided graphic data 138 in the area E is transferred. At that time, the generated graphic management unit 125 does not delete the divided graphic data 137 of the area E stored in the memory 124. When the graphic generation unit 122 completes the transfer of the divided graphic data 138 of the area E to the memory 108, the graphic generation unit 122 sends a generation completion notification 155 to the processor 107. Thereafter, the graphic generation unit 122 starts generating divided graphic data for the next region D.

  When the processor 107 receives notification of the completion of the generation of all the divided graphic data from the graphic generation units 106 and 122, the processor 107 instructs the graphic generation units 106 and 122 to stop generating the graphic data, and the synthesis unit 104. Instruct to start superimposing graphic data. The generation stop instruction 156 is an instruction for the processor 107 to instruct the graphic generation unit 106 to stop the generation of graphic data. The generation stop instruction 157 is an instruction for the processor 107 to instruct the graphic generation unit 122 to stop generation of graphic data.

  Next, the case of time T + 1 in FIG. 2, that is, the case where the display image data 203 and 204 are displayed will be described as an example. The graphic generation unit 106 does not generate the divided graphic data of the area C again because the divided graphic data of the area C already exists in the memory 108, and sends a generation completion notification 154 to the processor 107 again. Similarly, the graphic generation unit 106 does not generate the divided graphic data of the area D again because the divided graphic data of the area D already exists in the memory 108, and sends a generation completion notification 154 to the processor 107 again. Since the divided graphic data of the area F already exists in the memory 124, the graphic generation unit 122 does not generate the divided graphic data of the area F again and sends a generation completion notification 155 to the processor 107. Similarly, since the divided graphic data of the area E already exists in the memory 124, the graphic generation unit 122 does not generate the divided graphic data of the area E again and sends a generation completion notification 155 to the processor 107.

  Upon receiving notification of completion of generation of all the divided graphic data from the graphic generation units 106 and 122, the processor 107 instructs the graphic generation units 106 and 122 to stop the graphic data generation, and instructs the synthesis unit 104 to Instruct to start superimposing graphic data. As described above, at time T, the graphic generation unit 122 generates divided graphic data of the region E in the assigned image region of the image processing chip A 101 to which the graphic processing unit A 101 does not belong, and holds it in the memory 124. Therefore, at time T + 1, the graphic generation unit 122 does not need to perform divided graphic data generation and memory transfer for the region E again. As a result, it is possible to prevent the generation of the same divided graphic data from being performed more than necessary or the memory transfer between image processing chips from being performed more than necessary. That is, the processing load on the image processing chip can be reduced, and the display speed of graphic data can be improved.

  Next, a processing example of an image processing system that performs image processing by dividing image data into four image areas and assigning the four image areas to four separate image processing chips will be described with reference to FIG. As shown in FIG. 4, a case where one piece of high resolution image data is divided into four in a square shape will be described as an example. In FIG. 4, display image data 401, 402, 403, and 404 indicate display image data at time T that is the current frame, for example. Display image data 405, 406, 407, and 408 indicate display image data at time T + 1 of the next frame. FIG. 4 shows divided graphic data divided into regions C, D, E, F, G, H, I, J, K, L, M, and N. Further, it can be seen that the graphic data shown in FIG. 4 is translated to the right from time T to time T + 1.

  The generation order of the divided graphic data by the graphic generation unit belonging to the image processing chip in charge of the image area of the display image data 402 at time T will be described. Here, an arrow 4021 indicates the direction from the image boundary to the area division boundary in the rule 1 described above. The same applies to the arrows 4011, 4031, and 4041 in other image areas. The order determination unit 111 determines the generation order of the divided graphic data of the areas F and J from the rule 1 with respect to the graphic generation unit. Next, the order determination unit 111 determines the generation order of the divided graphic data of the areas E and I from the rule 2 with respect to the graphic generation unit. Further, the order determination unit 111 determines the generation order of divided graphic data of areas N, M, D, H, L, C, G, and K from the rule 3 with respect to the graphic generation unit. That is, the order determining unit 111 generates divided graphic data in the order of the areas F, J, E, I, N, M, D, H, L, C, G, and K to the graphic generating unit. The generation order is determined.

  The graphic generation unit divides the regions E and I in the image region of the display image data 401 in which the image processing chip different from the image processing chip to which the graphic generation unit belongs is in accordance with the generation order determined by the order determination unit 111. Generate graphic data. In this case, the graphic generation unit continues to hold the generated divided graphic data of the regions E and I in the memory of the image processing chip to which the graphic generation unit belongs, and to the memory of the image processing chip in charge of the image region of the display image data 401. Forward. Therefore, when the divided graphic data is generated in the image area of the display image data 406 at time T + 1, that is, in the same image area of the next frame, the memory of the image processing chip to which the graphic generation unit belongs already has the areas E and F. Split graphic data exists. Therefore, the graphic generation unit does not need to generate divided graphic data of the regions E and F again.

  If rule 2 described above is not set and the graphic generation unit generates divided graphic data in the order of regions F, J, M, N, I, E,. Think. Then, at time T, the graphic generation unit generates divided graphic data up to regions F, J, and M. In this case, at time T + 1, the graphic generation unit needs to generate divided graphic data of regions E and I. Therefore, the generation time for that is required. Further, even if the divided graphic data of the areas E and I stored in the memory of the other image processing chip is transferred to the memory of the image processing chip to which the graphic generation unit belongs, the transfer time is required. End up. As described above, by applying the above-described rule 2, it is possible to prevent the divided graphic data from being regenerated unnecessarily and the divided graphic data from being unnecessarily transferred between the image processing chips. . Here, the image area of the display image data 402 has been described as an example, but the same applies to other image areas.

  As described above, according to the present embodiment, in the image processing system including a plurality of image processing chips, the graphic data that translates the area division boundary of the area in charge of each image processing chip, that is, the boundary where the image data is divided into areas. Even in some cases, useless graphic data generation and useless memory transfer can be suppressed. This is not only the spatial information at a certain point of time, that is, the information indicating the assigned area in the space allocated to each image processing chip and the information indicating the display position of the graphic data, but also the information on the time axis, that is, in terms of time. This is because the generation order of the divided graphic data is determined in consideration of the graphic data before and after. This is because a mechanism for holding the generated divided graphic data on the memory is provided. Therefore, according to the present embodiment, the generation efficiency of graphic data can be improved, and the display speed of graphic data, that is, the update rate can be increased.

<Other embodiments>
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

  Note that the graphic data division unit is optimal depending on various factors such as the size of the graphic data, the processing capacity of the graphic generation unit, the overhead of communication between the graphic generation unit and the processor, and the bandwidth and latency during memory transfer. Different division units vary. Therefore, in the present invention, the division unit of graphic data is not particularly limited. In the above-described embodiment, the graphic data moving in the right direction is described as an example of the graphic data moving in parallel. However, the present invention can cope with the movement in the vertical and horizontal directions and in the diagonal direction. The moving direction is not limited.

  As mentioned above, according to each embodiment mentioned above, graphic data can be generated efficiently and the display speed of graphic data can be improved.

  The preferred embodiment of the present invention has been described in detail above, but the present embodiment is not limited to the specific embodiment, and various modifications can be made within the scope of the gist of the present invention described in the claims.・ Change is possible.

101: Image processing chip A, 102: Reception unit, 103: Image processing unit, 104: Composition unit, 105: Output unit, 106, 122: Graphic generation unit, 107: Processor, 108, 124: Memory, 109: Moving graphic Extraction unit, 110: Graphic division unit, 111: Order determination unit, 112: Holding instruction addition unit, 113, 125: Generated graphic management unit

Claims (10)

An image processing apparatus that performs processing in parallel by assigning the processing of each divided image data obtained by dividing image data constituting a moving image to different image processing units,
Extracting means for extracting graphic data that moves with time from the graphic data to be combined with the image data;
When the graphic data extracted by the extraction means moves and crosses the boundary of the divided image data in the preceding and following frames, dividing means for dividing the graphic data at a position intersecting the boundary at least in the preceding and following frames;
A determining unit that determines a generation order of a plurality of divided graphic data divided by the dividing unit based on movement information of the graphic data with respect to the graphic generating unit of the image processing unit;
An image processing apparatus.   When the graphic generation unit of the image processing unit that processes one divided image data generates the divided graphic data in the area of the other divided image data, the determining unit is configured to perform the one frame in the next frame of the divided graphic data. The image processing apparatus according to claim 1, wherein the generation order for the graphic generation unit is determined so as to be generated with priority from the divided graphic data moving to the divided image data area.   When notifying the generation command in the generation order to the graphic generation means of the image processing unit that processes the one divided image data, the area from the other divided image data is changed to the one divided image data area. The image processing apparatus according to claim 2, further comprising holding command adding means for adding a holding command to be held when moving divided graphic data is generated.   When the divided graphic data according to the generation command to which the holding instruction is added is generated by the graphic generation unit of the image processing unit that processes the one divided image data, the generated divided graphic data is converted into the one divided image data. The image processing apparatus according to claim 3, further comprising management means for storing and managing in a storage area of an image processing unit that processes image data and a storage area of an image processing unit that processes the other divided image data.   5. The image processing apparatus according to claim 1, wherein the extraction unit extracts the moving graphic data based on the movement information included in image information of graphic data generated by a user instruction. 6. .   The dividing unit specifies a position where the extracted graphic data intersects the boundary in the preceding and following frames based on movement amount information and movement direction information of the graphic data included in the movement information, and the graphic data The image processing device according to claim 1, wherein the image processing device is divided at least at the specified position.   The determination unit specifies the display position of the plurality of divided graphic data divided in the next frame based on movement amount information and movement direction information of the graphic data included in the movement information, thereby generating the graphic The image processing apparatus according to claim 1, wherein the generation order is determined for a unit.   When the graphic generation means receives a generation command in the generation order and the divided graphic data related to the generation instruction is already stored in the storage area of the image processing unit to which the graphic generation means belongs, The image processing apparatus according to claim 1, wherein the divided graphic data according to the generation instruction is not generated. A control method executed by an image processing apparatus that performs processing in parallel by allocating processing of each divided image data obtained by dividing image data constituting a moving image to different image processing units,
An extraction step of extracting graphic data that moves with time from the graphic data to be combined with the image data;
A division step of dividing the graphic data at a position intersecting the boundary at least in the preceding and following frames when the graphic data extracted in the extraction step moves and crosses the boundary of the divided image data in the preceding and following frames;
A determination step of determining a generation order of a plurality of divided graphic data divided by the dividing step based on movement information of the graphic data for a graphic generation means of the image processing unit;
Control method. A computer that controls an image processing apparatus that performs processing in parallel by assigning processing of each divided image data obtained by dividing image data constituting a moving image to different image processing units,
An extraction step of extracting graphic data that moves with time from the graphic data to be combined with the image data;
A division step of dividing the graphic data at a position intersecting the boundary at least in the preceding and following frames when the graphic data extracted in the extraction step moves and crosses the boundary of the divided image data in the preceding and following frames;
A determination step of determining a generation order of a plurality of divided graphic data divided by the dividing step based on movement information of the graphic data for a graphic generation means of the image processing unit;
A program for running

Images