JP2014107572A - Image data processing device and image data processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To synthesize a projection image and a comment and the like without allowing deterioration in the legibility of the synthesized image.SOLUTION: In the state of defining a specific position of a projection image 50 projected to a projected body as a first projection reference position 56, a specific position of a description image 70 described on the projected body, the position existing in the projection image, as first description reference positions 74,76, a position equivalent to the first projection reference position in a synthesized image 80 as a second projection reference position 52, and a position equivalent to the first description reference position in the synthesized image as a second description reference position 62, the projection image is synthesized with a description image in a state where: a relative position of the second description reference position to the second projection reference position is the same as a relative position of the first description reference position to the first projection reference position; or a relative position of the second description reference position to the first description reference position can be recognized. This enables appropriate synthesis of a projection image with a description image without allowing deterioration in legibility of the synthesized image.

Description

  The present invention relates to an image data processing apparatus and an image data processing program for processing image data.

  In meetings, presentations, etc., an image is generally projected onto a projection object such as a whiteboard using a projector, and explanations are given while referring to the projected image. At this time, a comment or the like is written on the projected image on the whiteboard. For this reason, in order to create a material reflecting the content of comments and the like written on the whiteboard, it is desired to combine the image projected on the whiteboard and the comments and the like written on the whiteboard. For this reason, in the image data processing device described in the following patent document, a description of a comment or the like is captured by a camera, and the data of the description image and the data corresponding to the projection image projected on the projection object are combined. Processing is in progress.

JP 2011-33936 A

  According to the image data processing device described in the above-mentioned patent document, it is possible to synthesize a projected image projected on a whiteboard and a comment or the like written on the whiteboard. However, when the projection image and the comment are combined, the relative position of the comment with respect to the projection image may be changed. In such a case, the projection image and the comment in the combined image may be changed. The positional relationship is different from the positional relationship between the projected image on the whiteboard and the comment. Since the change in the positional relationship between the projected image and the comment impairs the relevance between the image and the comment, the readability of the synthesized image may be reduced. The present invention has been made in view of such circumstances, and provides a technique capable of appropriately combining a projection image and a comment or the like without degrading the legibility of the combined image.

  In order to solve the above problems, an image data processing apparatus according to the present invention includes an image data storage unit that stores image data, and a projection image based on the image data stored in the image data storage unit. An image data processing apparatus comprising: an image projecting unit to project; and an image capturing unit that captures a description image described in the projection object onto which the projection image is projected and acquires description image data of the description image. Then, based on the description image data acquired by the imaging unit, at least a part of the description range in which the description image is described in the projection range in the projection target where the projection image is projected. A determination means for determining whether or not the image is not included; and when the determination means determines that the projection range does not include at least a part of the description range, Data combining means for generating combined image data obtained by combining the corresponding projection image data and the described image data, and combined data output means for outputting the combined image data combined by the data combining means, A specific position in the projection image projected onto the projection object is a first projection reference position and a specific position in the description image described on the projection object, and is projected onto the projection object. In addition, a position corresponding to the first projection reference position in the image based on the composite image data is a position that exists in the projection image, a second reference reference position, and the composite image data. In the case where the position corresponding to the first description reference position in the image based on the second description reference position is defined as the second description reference position, the data synthesis means 2. A state where the relative position of the second described reference position is the same as the relative position of the first described reference position with respect to the first projection reference position, or the first described reference position of the second described reference position The composite image data is generated in a state where a relative position with respect to can be recognized.

  In order to solve the above-described problem, an image data processing program according to the present invention includes an image data storage unit that stores image data, and a projection image based on the image data stored in the image data storage unit. An image data processing apparatus comprising: an image projecting unit that projects onto a body; and an imaging unit that captures a description image described on the projection object onto which the projection image is projected and acquires description image data of the description image The computer-readable image data processing program, wherein the description image is included in a projection range in the projection target in which the projection image is projected based on the description image data acquired by the imaging unit. A determination means for determining whether or not at least a part of the description range described is not included, and the determination means includes the description range in the projection range. When it is determined that at least a part of the data is not included, a data synthesis unit that generates synthesized image data obtained by synthesizing the projection image data corresponding to the projection image and the described image data, and a synthesis by the data synthesis unit A combined data output means for outputting the combined image data, the computer functioning, a specific position in the projection image projected on the projection object as a first projection reference position, and the projection target A specific position in the description image described on the body, and a position existing in the projection image projected on the projection object in the image based on the first description reference position and the composite image data , A position corresponding to the first projection reference position is a second projection reference position, and a position corresponding to the first description reference position in an image based on the composite image data is a second description reference position. In the case where each is defined, the data synthesizing unit is configured such that the relative position of the second described reference position with respect to the second projection reference position is relative to the first projection reference position. The synthesized image data is generated in a state that is the same as the current position, or in a state where the relative position of the second described reference position with respect to the first described reference position can be recognized.

  In the image data processing apparatus and the image data processing program configured as described above, the relative position of the second described reference position with respect to the second projection reference position is the relative position of the first described reference position with respect to the first projection reference position. The projection image data and the description image data are synthesized in the same state as the position. That is, the positional relationship between the specific position of the projection image and the specific position of the description image in the synthesized image is the positional relationship between the specific position of the projection image and the specific position of the description image on the projection object. It is the same. Thereby, it is possible to synthesize the projection image and the description image while maintaining the positional relationship between the projection image and the description image on the whiteboard. Further, the projection image data and the description image data are synthesized in a state where the relative position of the second description reference position to the first description reference position can be recognized. That is, in the synthesized image, it is possible to recognize a specific position of the description image on the whiteboard and a specific position of the description image in the composite image. As a result, it is possible to recognize a specific position of the description image before composition in the composite image. Therefore, according to the image data processing apparatus and the image data processing program of the present invention, it is possible to appropriately synthesize the projected image and the description image without degrading the legibility of the synthesized image.

1 is a block diagram of a projector 10. FIG. FIG. 4 is a diagram showing an operation flowchart of the projector 10. FIG. 4 is a diagram showing an operation flowchart of the projector 10. FIG. 4 is a diagram showing an operation flowchart of the projector 10. FIG. 4 is a diagram showing an operation flowchart of the projector 10. FIG. 4 is a diagram showing an operation flowchart of the projector 10. FIG. 4 is a diagram showing an operation flowchart of the projector 10. FIG. 3 is a diagram showing an operation flowchart of the projector 10 FIG. 3 is a diagram showing an operation flowchart of the projector 10 It is a figure which shows the projection image projected by the projector, and the description image described by the user. It is a figure which shows the image which synthesize | combined the projection image and the description image according to the expansion synthesis method. It is a figure which shows the image which synthesize | combined the projection image and the description image according to the expansion synthesis | combination method, and was further contracted. It is a figure which shows the image which synthesize | combined the projection image and the description image according to the shrinkage | contraction synthesis | combination method. It is a figure which shows the image which synthesize | combined the projection image and the description image according to the movement synthetic | combination method. It is a figure which shows the image which synthesize | combined the projection image and the description image according to the newline composition method. It is a figure which shows the image which synthesize | combined the projection image and two description images according to the expansion synthesis method. It is a figure which shows the image which synthesize | combined the projection image and the description image according to the shrinkage | contraction synthesis | combination method, and was synthesize | combined the projection image and the description image according to the movement synthesis method.

<Configuration of projector>
FIG. 1 shows a block diagram of a projector 10 exemplified as an embodiment according to the present application. The projector 10 mainly includes a CPU (abbreviation of Central Processing Unit) 12, a storage unit 14, a liquid crystal panel 16, a light source lamp 18, a projection lens 20, a camera 22, and a USB I / F 24. These components can communicate with each other via the input / output port 26.

  The liquid crystal panel 16 is built in the projector 10 and displays an image. The light source lamp 18 transmits light toward the liquid crystal panel 16 on which an image is displayed. The projection lens 20 enlarges and projects the light transmitted through the liquid crystal panel 16 toward a projection object such as a white board or a screen. With such a configuration, the projector 10 projects an image on the projection target.

  The camera 22 is disposed toward the projection target, and captures a description image such as characters written on the projection target. The USB I / F 24 can be connected to a USB memory, a PC (abbreviation of personal computer), and the like. Thereby, the projector 10 can obtain image data from a USB memory, a PC, or the like.

  The storage unit 14 stores an image data processing program 28. The image data processing program 28 is a program for causing the CPU 12 to execute processing for synthesizing image data corresponding to an image projected on the projection object and imaging data obtained by imaging by the camera 22. .

  In addition, the storage unit 14 includes a data storage area 30. The data storage area 30 is an area for storing image data acquired from a USB memory, a PC, or the like via the USB I / F 24, imaging data obtained by imaging by the camera 22, and the like.

  The CPU 12 executes processing according to the image data processing program 28 in the storage unit 14. Hereinafter, the CPU 12 that executes the image data processing program 28 may be simply described as a program name. For example, the description “the image data processing program 28 is” may mean “the CPU 12 that executes the image data processing program 28”. The storage unit 14 is configured by combining a RAM (abbreviation of random access memory), a ROM (abbreviation of read only memory), a flash memory, an HDD (abbreviation of hard disk), a buffer provided in the CPU 12, and the like.

<Projector operation>
In conferences and presentations, an image is generally projected onto a projection object such as a whiteboard using the projector 10, and explanations are made while referring to the projected image. At this time, handwritten information such as comments is written on the whiteboard on the projected image. For this reason, in order to create a document reflecting the contents of the handwritten information written on the whiteboard, it is desired to combine the image projected on the whiteboard and the handwritten information written on the whiteboard.

  The user performs marking indicating the projection range of the projection image to be synthesized. An image projected on a whiteboard or the like is usually rectangular. The user makes a pair of opposite corners of the rectangular image, specifically, as shown in FIG. 10, the first corner portion 52 located at the upper left of the image and the second corner portion 54 located at the lower right. At the same time, two magnet buttons 56 and 57 are attached. The projector 10 specifies the projection range 50 of the projection image from the attachment positions of the magnet buttons 56 and 57.

  Further, the description range of the handwritten information to be synthesized is specified by the same method as the projection range, that is, by sticking the magnet button. Specifically, as shown in FIG. 10, the user has two magnet buttons on the third corner 62 located at the upper left of the symbol 60 as handwritten information and the fourth corner 64 located at the lower right. 66 and 67 are pasted. Then, the projector 10 specifies the description range 70 of the symbol 60 from the attachment position of the magnet buttons 66 and 67.

  When the projection range 50 and the description range 70 are specified by the above method, the projector 10 determines whether or not at least a part of the description range 70 is not included in the projection range 50. That is, the projector 10 determines whether or not at least a part of the description range 70 protrudes from the projection range 50. When it is determined that the description range 70 does not protrude, that is, when the entire description range 70 is within the projection range 50, the relative position of the description range 70 within the projection range 50 is maintained. Then, the image data of the projection image and the imaging data of the handwritten information are combined.

  On the other hand, when it is determined that the description range 70 protrudes, the imaging data or the image data is processed so that the entire description range 70 falls within the projection range 50. In the projector 10, five types of processing methods are prepared as processing methods for imaging data or image data, and the user can select from these five types of processing methods.

  Specifically, five types of magnet buttons for specifying the description range 70 are prepared according to the five types of processing methods. Specifically, quadrangular magnet buttons 66 and 67 (see FIG. 11), triangular magnet buttons 72 and 73 (see FIG. 12), star-shaped magnet buttons 74 and 75 (see FIG. 13), and pentagon-shaped magnet buttons 72 and 73 (see FIG. 12). Magnet buttons 76 and 77 (see FIG. 14) and hexagonal magnet buttons 78 and 79 (see FIG. 15) are prepared. And what was selected by the user among five types of magnet buttons 66, 67, 72, 73, 74, 75, 76, 77, 78, 79 is stuck on the whiteboard. The processing method according to the magnet button is adopted. Thereby, the user can select a processing method by a simple method. In the projector 10, magnet buttons (not shown) other than the above five types of magnet buttons 66, 67, 72, 73, 74, 75, 76, 77, 78, 79 are also prepared. When pasted on the board, a method preset in the projector 10 is employed. Hereinafter, five types of processing methods will be described.

  The first processing method is a method of extending the projection range 50 in the synthesized image by adding margin data to the image data. Specifically, as illustrated in FIG. 11, the projector 10 adds margin data to the image data so as to include a portion corresponding to a part of the description range 70 that protrudes from the projection range 50. Thereby, the projection range 50 in the composite image 80 (the figure shown on the right side) is expanded in the direction toward the description range 70 with the first corner 52 as a reference. Then, the projector 10 combines the processed image data and the imaging data in a state where the positional relationship between the first corner portion 52 of the projection range 50 and the third corner portion 62 of the description range 70 is maintained.

  In the method in which data is synthesized after the projection range 50 is expanded, the projection range 50 is expanded on the basis of the first corner 52 of the projection range 50. Therefore, the first corner 52 that defines the projection range 50; The positional relationship with the third corner 62 that defines the description range 70 does not change before and after the synthesis. That is, the relative position of the magnet button 66 with respect to the magnet button 56 on the whiteboard and the relative position of the third corner 62 with respect to the first corner 52 in the composite image 80 are not changed. Thereby, it becomes possible to synthesize the projected image and the handwritten information without impairing the relevance between the projected image and the handwritten information. The magnet buttons corresponding to the method according to the extended synthesis are the square magnet buttons 66 and 67.

  The second processing method is a method of reducing the number of pixels of data synthesized by the method according to the above-described extended synthesis. Specifically, in the method according to the extended synthesis, the number of pixels of the synthesized data is larger than the number of pixels of the image data before synthesis by adding margin data to the image data. For this reason, the projector 10 performs a process for reducing the number of pixels of the synthesized data. For example, in the method according to the extension synthesis in FIG. 11, the projection range 50 is extended in the width direction. For this reason, the projector 10 processes the combined data so that the number of pixels in the width direction in the combined data becomes the number of pixels in the width direction in the image data before the expansion processing. As a result, the number of pixels of the composite image 80 is reduced, and the composite image 80 contracts as shown in FIG. As a result, the data amount can be reduced. Note that the magnet buttons corresponding to the method of reducing the number of pixels after the extended synthesis are triangular magnet buttons 72 and 73.

  The third processing method is a method of shrinking the description range 70 in the synthesized image by reducing the number of pixels of the imaging data. Specifically, as shown in FIG. 13, the projector 10 reduces the number of pixels of the imaging data so that a part of the description range 70 that protrudes from the projection range 50 enters the projection range 50. Thereby, the description range 70 in the composite image 80 (the figure shown on the right side) contracts in the direction toward the projection range 50 with the third corner 62 as a reference. Then, the projector 10 combines the image data and the processed imaging data while maintaining the positional relationship between the first corner 52 of the projection range 50 and the third corner 62 of the description range 70.

  In the method in which data is synthesized after the description range 70 is contracted, the description range 70 is contracted with reference to the third corner 62 of the description range 70, and therefore, the first corner 52 that defines the projection range 50; The positional relationship with the third corner 62 that defines the description range 70 does not change before and after the synthesis. That is, the relative position of the magnet button 74 with respect to the magnet button 56 on the whiteboard and the relative position of the third corner portion 62 with respect to the first corner portion 52 in the composite image 80 are not changed. Thereby, it becomes possible to synthesize the projected image and the handwritten information without impairing the relevance between the projected image and the handwritten information. The magnet buttons corresponding to the method according to the contraction synthesis are the star-shaped magnet buttons 74 and 75.

  In addition, if the shrinkage rate in the description range 70 using a method according to shrinkage synthesis becomes too high, there is a possibility that the written content cannot be understood. For this reason, when the shrinkage rate of the description range 70 in the method according to the shrinkage synthesis exceeds a predetermined shrinkage rate, it is possible to adopt a different method from the method according to the shrinkage synthesis. Thereby, it is possible to avoid a situation in which the description content is not understood due to the contraction of the description range 70. Note that, when the handwritten information described in the description range 70 is contracted by the method according to the contraction synthesis, and the size of the contracted handwritten information becomes smaller than a predetermined size, the method according to the contraction synthesis Different methods may be used.

  The fourth processing method is a method of moving the relative position of the description range 70 with respect to the projection range 50. Specifically, the projector 10 detects the blank portion in the projection range 50 based on the image data, and processes the imaging data so as to move the description range 70 to the blank portion. Then, the processed imaging data and image data are synthesized. Furthermore, the broken line data connecting before and after the movement of the description range 70 is added to the synthesized data. Specifically, as shown in FIG. 14, the projector 10 is a broken line that connects the position in the composite image 80 corresponding to the attachment position of the magnet button 76 and the third corner 62 of the description range 70 in the composite image 80. 82 data is added to the synthesized data. This broken line 82 makes it possible to associate the position of the handwritten information after the movement with the position of the handwritten information before the movement, and recognizes where the handwritten information was written in the projected image before combining the data. It becomes possible. The magnet buttons corresponding to the method according to the movement composition are pentagonal magnet buttons 76 and 77.

  The fifth processing method is a method for causing a line break in a character string when the handwritten information is a character string. Specifically, as shown in FIG. 15, the projector 10 sets the description range 70 existing outside the projection range 50 to characters existing inside and outside the projection range 50 for each line of the character string. And a line feed designation range 86. Then, for each line of the character string, the imaging data is processed so that the line feed designation range 86 existing outside the projection range 50 is positioned below the line feed designation range 86 existing within the projection range 50. However, the line feed of the character string is performed with the third corner 62 as a reference. In other words, the projector 10 maintains the position of the line feed designation range 86 including the third corner 62, and sequentially makes another line feed designation range 86 below the maintained line feed designation range 86. Then, the image data and the processed imaging data are combined while maintaining the positional relationship between the first corner 52 of the projection range 50 and the third corner 62 of the description range 70.

  In the technique in which imaging data and image data in which a character string is broken are combined, the character string is broken with reference to the third corner portion 62 of the description range 70. Therefore, the first corner that defines the projection range 50 is used. The positional relationship between the portion 52 and the third corner portion 62 that divides the description range 70 does not change before and after the synthesis. That is, the relative position of the magnet button 78 with respect to the magnet button 56 on the whiteboard and the relative position of the third corner 62 with respect to the first corner 52 in the composite image 80 are not changed. Thereby, it becomes possible to synthesize the projected image and the handwritten information without impairing the relevance between the projected image and the handwritten information. The magnet buttons corresponding to the method according to the line feed composition are hexagonal magnet buttons 78 and 79.

<Image data processing program>
The composition process of the image data and the imaging data is performed by the image data processing program 28 being executed by the CPU 12. Hereinafter, a flow for synthesizing the projection image and the handwritten information will be described with reference to FIGS. 2 to 9.

  First, in S100 of the main flow shown in FIG. 2, the CPU 12 determines whether or not the composition of the projected image projected on the whiteboard and the handwritten information written on the whiteboard is requested. Specifically, the projector 10 is provided with a synthesis start button (not shown) for starting the synthesis process of the projection image and the handwritten information, and the CPU 12 determines whether or not the synthesis start button has been operated. to decide. When the synthesis start button is not operated (S100: NO), that is, when the synthesis of the projection image and the handwritten information is not requested, the process of S100 is repeated.

  On the other hand, when the composition of the projection image and the handwritten information is requested (S100: YES), the process proceeds to S102. In S102, the CPU 12 turns off the light source lamp 18 and stops projecting the projection image on the whiteboard. Then, the process proceeds to S104.

  In S <b> 104, the CPU 12 transmits an imaging command to the camera 22. Thereby, the camera 22 images the handwritten information written on the whiteboard. Then, the process proceeds to S106. In S <b> 106, the CPU 12 determines whether the marker for specifying the projection range of the projection image projected on the whiteboard and the marker for specifying the description range of the handwritten information described on the whiteboard have been recognized. to decide.

  Specifically, the CPU 12 captures whether or not the magnet buttons 56, 57, 66, 67, 72, 73, 74, 75, 76, 77, 78, and 79 are attached to the whiteboard by the camera 22. Judgment based on data. In this determination, the presence / absence of the magnet button image data in the imaging data is determined using the shape, color, etc. of the magnet button. If the magnet button cannot be recognized (S106: NO), the process proceeds to S108. In S108, the CPU 12 projects an image for indicating an error occurrence on the whiteboard. Then, the main flow ends.

  On the other hand, if the magnet button can be recognized (S106: YES), the process proceeds to S110. In S110, the CPU 12 performs processing in accordance with the synthesis processing flow shown in FIG. In the composition processing flow, in S120, the CPU 12 specifies the description range 70 of the symbol 60 from the attaching position of the magnet buttons 66, 67, 72, 73, 74, 75, 76, 77, 78, 79. Then, the process proceeds to S122.

  In S <b> 122, the CPU 12 specifies the projection range 50 of the projection image from the attachment positions of the magnet buttons 56 and 57. Then, the process proceeds to S124. In S <b> 124, the CPU 12 determines whether the description range 70 exists outside the projection range 50. That is, it is determined whether or not at least a part of the description range 70 is not within the projection range 50. When the description range 70 does not exist outside the projection range 50 (S124: NO), the process proceeds to S126. In S <b> 126, the CPU 12 combines the imaging data and the image data in a state where the relative position of the description range 70 within the projection range 50 is maintained. Then, the synthesis process flow is terminated, and the process returns to the main flow.

  On the other hand, when the description range 70 exists outside the projection range 50 (S124: YES), the process proceeds to S128. In S128, the CPU 12 determines whether or not the magnet button for specifying the description range 70 is a magnet button corresponding to the user designation. Specifically, the CPU 12 determines whether or not the magnet buttons 66, 67, 72, 73, 74, 75, 76, 77, 78, and 79 are attached to the whiteboard based on the imaging data. If magnet buttons 66, 67, 72, 73, 74, 75, 76, 77, 78, 79 are stuck on the whiteboard (S128: YES), the process proceeds to S130. In S130, the CPU 12 performs a process according to the marker determination flow shown in FIG.

  In the marker determination flow, in S140, the CPU 12 determines whether or not a method according to extended synthesis is designated. Specifically, it is determined whether or not the magnet buttons 66, 67, 72, 73 are attached to the whiteboard. When the magnet buttons 66, 67, 72, 73 are stuck on the whiteboard (S140: YES), the process proceeds to S142. In S142, the CPU 12 performs a process according to the extended synthesis flow shown in FIG.

  In the extended synthesis flow, in S160, the CPU 12 adds margin data to the image data and extends the projection range 50 as described above. Then, the process proceeds to S162. In S <b> 162, the CPU 12 combines the imaging data and the image data in which the projection range 50 is extended while maintaining the positional relationship between the first corner 52 of the projection range 50 and the third corner 62 of the description range 70. To do. Thereby, a composite image 80 (shown on the right side) as shown in FIG. 11 is obtained. Then, the process proceeds to S164. In S164, the CPU 12 determines whether or not a contraction instruction is given by the marker.

  Specifically, the CPU 12 determines whether or not the magnet buttons 72 and 73 are attached to the whiteboard. When the magnet buttons 72 and 73 are not attached to the whiteboard, that is, when the shrinkage instruction is not given (S164: NO), the extended synthesis flow is terminated and the process returns to the marker determination flow.

  On the other hand, if the magnet buttons 72 and 73 are adhered to the whiteboard, that is, if the shrinkage instruction is given (S164: YES), the process proceeds to S166. In S166, the CPU 12 processes the combined data so that the number of pixels of the combined image becomes the number of pixels of the image before combining as described above. As a result, the number of pixels of the composite image 80 decreases, and the composite image 80 contracts as shown in FIG. Then, the extended synthesis flow is terminated, and the flow returns to the marker determination flow.

  On the other hand, in S140 of the marker determination flow, when the method according to the extended synthesis is not designated (S140: NO), the process proceeds to S144. In S144, the CPU 12 determines whether or not a method according to contraction synthesis is designated. Specifically, it is determined whether or not the magnet buttons 74 and 75 attached to the whiteboard are attached. If the magnet buttons 74 and 75 are attached to the whiteboard (S144: YES), the process proceeds to S146. In S146, the CPU 12 performs processing according to the contraction synthesis flow shown in FIG.

  In the shrinkage synthesis flow, in S170, the CPU 12 detects a group of handwritten information written on the whiteboard. Specifically, handwritten information may be separately described in a plurality of parts of the wide board, and in such a case, there are a plurality of individual description ranges. For this reason, in S170, CPU12 detects one or more description ranges as a group of handwritten information. Then, the process proceeds to S172.

  In S172, the CPU 12 processes the imaging data so as to contract the description range 70 that exists outside the projection range 50 in the group of handwritten information. Specifically, the description range 70 that exists outside the projection range 50 out of the one or more description ranges 70 is extracted. Then, as described above, processing for reducing the number of pixels is performed on the extracted imaging data in the description range 70. As a result, the description range 70 contracts so as to be within the projection range 50. Then, the process proceeds to S174.

  In S174, the CPU 12 combines the image data and the imaging data in which the description range 70 is contracted while maintaining the positional relationship between the first corner 52 of the projection range 50 and the third corner 62 of the description range 70. To do. Thereby, a composite image 80 (shown on the right side) as shown in FIG. 13 is obtained. Then, the contraction synthesis flow is terminated, and the flow returns to the marker determination flow.

  On the other hand, in S144 of the marker determination flow, if the process according to contraction synthesis is not employed (S144: NO), the process proceeds to S148. In S <b> 148, the CPU 12 determines whether or not a technique according to move synthesis is designated. Specifically, it is determined whether or not magnet buttons 76 and 77 are attached to the whiteboard. When the magnet buttons 76 and 77 are stuck on the whiteboard (S148: YES), the process proceeds to S150. In S150, the CPU 12 performs processing in accordance with the movement synthesis flow shown in FIG.

  In the movement synthesis flow, in S180, the CPU 12 detects a blank portion of the projection image based on the image data. Then, the process proceeds to S182. In S182, the CPU 12 detects a group of handwritten information written on the whiteboard. Then, the process proceeds to S184. In S <b> 184, the CPU 12 processes the imaging data so as to move the description range 70 existing outside the projection range 50 in the group of handwritten information to the blank portion of the image. Specifically, the description range 70 that exists outside the projection range 50 out of the one or more description ranges 70 is extracted. Then, the imaging data is processed so that the extracted description range 70 moves to the blank portion of the image. Then, the process proceeds to S186.

  In S186, the CPU 12 synthesizes the image data and the processed imaging data. Then, the process proceeds to S188. In S188, as described above, the CPU 12 adds, to the synthesized data, broken line data connecting before and after the movement of the description range 70. Then, the move synthesis flow is terminated, and the flow returns to the marker determination flow.

  On the other hand, in S148 of the marker determination flow, when the method according to the move synthesis is not designated (S148: NO), the process proceeds to S152. In S152, the CPU 12 performs processing according to the line feed composition flow shown in FIG.

  In the line feed composition flow, in S190, the CPU 12 detects a group of handwritten information written on the whiteboard. Then, the process proceeds to S192. In S <b> 192, the CPU 12 determines whether or not at least one of the handwritten information group, that is, at least one of the one or more description ranges 70 exists outside the projection range 50. When at least one of the one or more description ranges 70 exists outside the projection range 50 (S192: YES), the process proceeds to S194.

  In S194, the CPU 12 determines whether or not the handwritten information existing outside the projection range 50 is a character string. In this determination, the presence / absence of character data in the imaging data is determined. When the handwritten information existing outside the projection range 50 is a character string (S194: YES), the process proceeds to S196. In S196, as shown in FIG. 15, the CPU 12 converts the description range 70 existing outside the projection range 50 into a character existing outside the projection range 50 and a character existing outside the projection range 50 for each line of the character string. In addition, the line break is divided using the line feed designation range 86. Then, the process proceeds to S198.

  In S <b> 198, the CPU 12 captures the imaging data so that the line feed designation range 86 existing outside the projection range 50 is positioned below the line feed designation range 86 existing within the projection range 50 for each line of the character string. To process. At this time, a line feed of the character string is performed with the third corner 62 as a reference. Then, the process proceeds to S200.

  In S <b> 200, the CPU 12 combines the image data and the processed imaging data while maintaining the positional relationship between the first corner 52 of the projection range 50 and the third corner 62 of the description range 70. Thereby, a composite image 80 (shown on the lower right side) as shown in FIG. 15 is obtained. Then, the line feed composition flow is terminated, and the flow returns to the marker determination flow.

  When at least one of the one or more description ranges 70 does not exist outside the projection range 50 in S192 (S192: NO), and when the handwritten information existing outside the projection range 50 is not a character string in S194 (S194: NO) ends the line feed composition flow and returns to the marker determination flow.

  In S128 of the composition processing flow, when the magnet buttons 66, 67, 72, 73, 74, 75, 76, 77, 78, 79 are not attached to the whiteboard, that is, to specify the description range 70. If the magnet button is not a magnet button corresponding to the user designation (S128: NO), the process proceeds to S132. In S <b> 132, the CPU 12 performs a process according to the synthesis process automatic determination flow shown in FIG. 9.

  In the composition processing automatic determination flow, in S210, the CPU 12 detects a group of handwritten information written on the whiteboard. Then, the process proceeds to S212. In S212, when the CPU 12 contracts the description range 70 that exists outside the projection range 50 in the group of handwritten information by a method according to contraction synthesis, is the contraction rate equal to or less than a predetermined contraction rate? Judge whether or not.

  When the contraction rate when the description range 70 is contracted is equal to or less than the predetermined contraction rate (S212: YES), the process proceeds to S214. In S214, the same processing as S172 is performed. Then, the process proceeds to S216. In S216, the same processing as S174 is performed. Then, the synthesis process automatic determination flow is terminated, and the process returns to the synthesis process flow.

  On the other hand, when the contraction rate when the description range 70 is contracted exceeds the predetermined contraction rate (S212: NO), the process proceeds to S218. In S218, the same process as in S180 is performed. Then, the process proceeds to S220. In S220, the same process as S184 is performed. Then, the process proceeds to S222. In S222, the same processing as S186 is performed. Then, the process proceeds to S224. In S224, the same processing as S188 is performed. Then, the synthesis process automatic determination flow is terminated, and the process returns to the synthesis process flow.

  In the compositing process flow, when the compositing process between the image data and the imaged data ends, the compositing process flow ends and the process returns to the main flow. In the main flow, the process proceeds to S112 after the synthesis processing flow ends. In S <b> 112, the CPU 12 determines whether or not the composition process of the image data and the imaging data has succeeded.

  If the image data and imaging data have been successfully combined (S112: YES), the process proceeds to S114. In S114, the CPU 12 projects an image based on the combined data on the whiteboard. Then, the main flow ends. On the other hand, when the composition processing of the image data and the imaging data is not successful (S112: NO), the process proceeds to S108. In S108, the CPU 12 projects an image for indicating an error occurrence on the whiteboard. Then, the main flow ends.

<Modification>
In the projector 10 according to the present embodiment, the projection range 50 and the description range 70 are specified by the shape of the magnet button, but the projection range 50 and the description range 70 are specified by a pattern or the like printed on the magnet button. Is possible. It should be noted that the shape, pattern, etc. of the magnet button can be a complicated shape rather than a simple shape such as a triangle. By making the shape, pattern, and the like of the magnet button complex, it is possible to reduce the possibility of misidentification between a handwritten image or a part of the projected image and the magnet button. In addition, the projection range and the like can be specified by various means without being limited to the specification of the projection range and the like by the magnet button. Specifically, for example, the projection range 50 and the description range 70 may be specified by drawing a line or the like described on a whiteboard.

  In the projector 10 according to the present embodiment, image data stored in the data storage area 30 is used as data corresponding to the projection image. However, the camera 22 captures the projection image projected on the whiteboard. In addition, the captured image data can be used as data corresponding to the projection image.

  In the projector 10 according to the present embodiment, the image based on the synthesized data is projected on the whiteboard, but the synthesized data can be output by various means. Specifically, for example, an image based on the combined data may be printed by a printer or the like, and an image based on the combined data can be displayed on a panel or the like. Alternatively, the synthesized data may be transmitted to another device and output.

  Further, in the projector 10 according to the present embodiment, the case where one description range 70 is outside the projection range 50 has been described, but the above description also applies when a plurality of description ranges 70 are outside the projection range 50. It is possible to adopt a technique. Specifically, for example, as illustrated in FIG. 16, when the two description ranges 70 are outside the projection range 50, the two description ranges 70 and the projection range 50 are synthesized by a technique according to extended synthesis. Therefore, it is necessary to process the image data so that the two described ranges 70 are within the projection range 50. Specifically, one description range 70 protrudes to the right of the projection range 50, and another one description range 70 protrudes below the projection range 50. Therefore, the projection range 50 is expanded rightward and downward with the first corner portion 52 as a reference. Incidentally, the projection range 50 needs to be expanded so that the two description ranges 70 are included in the projection range 50.

  Further, for example, when a plurality of description ranges 70 are outside the projection range 50, one description range 70 of the plurality of description ranges 70 and the projection range 50 are synthesized by a predetermined synthesis method. It is possible to synthesize another description range 70 of the description range 70 and the projection range 50 by a synthesis method different from the predetermined synthesis method. Specifically, for example, as illustrated in FIG. 17, when the two described ranges 70 are outside the projection range 50, one of the two described ranges 70 (the magnet button 74, The imaging data corresponding to the description range specified by 75 and the image data are synthesized by a method according to contraction synthesis, and the other description range (magnet buttons 76 and 77 in the figure) of the two description ranges 70 is combined. The imaging data corresponding to the description range specified by (1) and the image data may be synthesized by a technique according to the movement synthesis.

  In the projector 10 of the present embodiment, the case where the CPU 12 executed based on the image data processing program 28 performs various processes has been described. However, it is not limited to this form. The CPU 12 executed on the basis of the image data processing program 28 may issue an instruction for performing various processes to other systems and hardware configurations.

  In addition, the technical elements described in the present specification or the drawings exhibit technical usefulness alone or in various combinations, and are not limited to the combinations described in the claims at the time of filing. In addition, the technology illustrated in the present specification or the drawings achieves a plurality of objects at the same time, and has technical utility by achieving one of the objects.

  The projector 10 is an example of an image data processing device. The CPU 12 is an example of a computer. The storage unit 14 is an example of an image data storage unit. What is constituted by the liquid crystal panel 16, the light source lamp 18, and the projection lens 20 is an example of an image projection unit. The camera 22 is an example of an imaging unit. The image data processing program 28 is an example of an image data processing program. A whiteboard is an example of a projection target. The magnet buttons 66, 67, 72, 73, 74, 75, 76, 77, 78, 79 are examples of the outer edge display portion. Symbol 60 is an example of a description image. A broken line 82 is an example of a symbol. The position where the magnet button 56 is pasted on the whiteboard is an example of the first projection reference position. The position where the magnet buttons 66, 72, 74, 76, 78 are pasted on the whiteboard is an example of the first reference position. The position of the first corner 52 in the composite image 80 is an example of a second projection reference position. The position of the third corner 62 in the composite image 80 is an example of a second description reference position. The CPU 12 that executes S120 is an example of a description range specifying unit. The CPU 12 that executes S124 is an example of a determination unit. The CPU 12 that executes S132, S142, S146, S150, and S152 is an example of a data synthesizing unit. The CPU 12 that executes S114 is an example of a combined data output unit. The CPU 12 that executes S140, S144, and S148 is an example of a synthesis method determination unit. The CPU 12 that executes S142 is an example of an extended synthesis unit. The CPU 12 that executes S146 is an example of contraction synthesis means. The CPU 12 that executes S150 is an example of a movement synthesis unit. The CPU 12 that executes S152 is an example of a line feed composition unit. Image data is an example of projection image data. The imaging data is an example of description image data.

  Each program may be composed of a single program module, or may be composed of a plurality of program modules. In addition, each example may have another replaceable configuration and is within the scope of the present invention. It may be a computer (CPU 12) that executes processing based on the image data processing program 28, or may be a computer that executes processing based on programs other than the image data processing program, such as an operating system, other applications, and programs. It may be a hardware configuration that operates according to instructions from the computer, or may be a configuration in which the computer and the hardware configuration are linked. Of course, it may be a computer that executes processes by linking processes based on a plurality of programs, or may have a hardware configuration that operates according to instructions from a computer that executes processes by linking processes based on a plurality of programs. Good.

  10: projector, 12: CPU, 14: storage unit, 16: liquid crystal panel, 18: light source lamp, 20: projection lens, 22: camera

Claims (13)

An image data storage unit for storing image data;
An image projection unit that projects a projection image based on the image data stored in the image data storage unit onto a projection object;
An imaging unit that captures a description image described in the projection object on which the projection image is projected, and acquires description image data of the description image;
An image data processing apparatus comprising:
Based on the description image data acquired by the imaging unit, at least a part of the description range in which the description image is described is included in a projection range in the projection target on which the projection image is projected. A judging means for judging whether or not there is,
When the determination unit determines that at least a part of the description range is not included in the projection range, composite image data obtained by combining the projection image data corresponding to the projection image and the description image data is obtained. Data synthesizing means to generate;
Synthesized data output means for outputting the synthesized image data synthesized by the data synthesizing means;
With
A specific position in the projection image projected on the projection object is a first projection reference position,
A specific position in the description image described in the projection object, and a position existing in the projection image projected on the projection object is a first description reference position;
A position corresponding to the first projection reference position in the image based on the composite image data is a second projection reference position;
A position corresponding to the first description reference position in an image based on the composite image data is a second description reference position;
When defined,
The data synthesizing means includes
The relative position of the second description reference position with respect to the second projection reference position is the same as the relative position of the first description reference position with respect to the first projection reference position, or the second description The composite image data is generated in a state where a relative position of a reference position with respect to the first described reference position can be recognized. The data synthesizing means includes
The relative position of the second description reference position with respect to the second projection reference position is the same as the relative position of the first description reference position with respect to the first projection reference position, and the first projection reference position 2. The image data according to claim 1, further comprising: an extended composition unit that synthesizes the projection image data in which the projection range is expanded in a direction toward the description range with reference to the description image data. Processing equipment. When there are a plurality of the description images described in the projection object, and a range in which each of the plurality of description images is defined as an individual description range,
The extended synthesis means includes
The projection image data obtained by extending the projection range in a direction toward the one that protrudes most from the projection range among the plurality of individual description ranges, and the description image data are combined. 2. The image data processing apparatus according to 2. The extended synthesis means includes
One individual description range out of the plurality of individual description ranges protrudes in one of the vertical directions of the projection range, and one individual description range different from the one individual description range is in the horizontal direction of the projection range. 4. The projection image data obtained by extending the projection range to one of the up and down directions and one of the left and right directions and the description image data when they are overhanging. The image data processing device described in 1. The extended synthesis means includes
5. The process according to claim 2, further comprising: performing a process of reducing the number of pixels of the combined image data after combining the projected image data with the extended projection range and the description image data. The image data processing device described in 1. The data synthesizing means includes
The relative position of the second description reference position with respect to the second projection reference position is the same as the relative position of the first description reference position with respect to the first projection reference position, and the first description reference position 6. The method according to claim 1, further comprising contraction combining means for combining the description image data in which the description range is contracted in a direction toward the projection range with reference to the projection image data. 2. An image data processing apparatus according to item 1. The data synthesizing means includes
When the description image is a character string, a relative position of the second description reference position with respect to the second projection reference position is defined as a relative position of the first description reference position with respect to the first projection reference position. 7. A line feed synthesizing unit for synthesizing the description image data obtained by changing the character string to the line feed with the first description reference position as a reference and the projection image data. The image data processing device according to any one of the above. The data synthesizing means includes
The composite image data is generated in a state where the relative position of the second described reference position with respect to the second projection reference position moves from the relative position of the first described reference position with respect to the first projection reference position. Having mobile synthesis means,
The migration synthesis means includes
By synthesizing the synthesized image data and the symbol data of the symbol connecting the first written reference position and the second written reference position, the second written reference position is relative to the first written reference position. 8. The image data processing apparatus according to claim 1, wherein the position can be recognized. The data synthesizing means includes
The relative position of the second description reference position with respect to the second projection reference position is the same as the relative position of the first description reference position with respect to the first projection reference position, and the first description reference position And the contraction synthesis means for synthesizing the description image data in which the description range is contracted in the direction toward the projection range, and the projection image data,
9. The image data processing apparatus according to claim 8, wherein when the contraction rate of the described range in the contraction combining unit exceeds a predetermined contraction rate, the combined image data is generated according to the movement combining unit. When there are a plurality of the description images described in the projection object, and a range in which each of the plurality of description images is defined as an individual description range,
The data synthesizing means includes
The description image data and the projection image data corresponding to all of the plurality of the individual description ranges that are included in the projection range are maintained in a relative position between the projection range and the individual description range. Synthesized in the state,
Of the plurality of individual description ranges, the description image data and the projection image data corresponding to those in which at least a part is not included in the projection range, the second description reference position with respect to the second projection reference position A state in which the relative position is the same as the relative position of the first written reference position with respect to the first projection reference position, or the relative position of the second written reference position with respect to the first written reference position The image data processing apparatus according to claim 1, wherein the image data processing apparatus combines the images in a recognizable state. When there are a plurality of the description images described in the projection object, and a range in which each of the plurality of description images is defined as an individual description range,
The data synthesizing means includes
The description corresponding to one individual description range of the two or more individual description ranges when at least a part of two or more individual description ranges of the plurality of individual description ranges are not included in the projection range. The image data and the projection image data are synthesized according to a predetermined method, and the description image data corresponding to the one individual description range and the one individual description range of the two or more individual description ranges, and the The image data processing apparatus according to claim 1, wherein the projection image data is synthesized according to a technique different from the predetermined technique. A description range specifying means for specifying the description range by means of an outer edge display unit indicating an outer edge of the description image;
A synthesis method determining means for determining a synthesis method of the description image data and the projection image data according to a display mode of the outer edge display unit;
The image data processing apparatus according to claim 1, further comprising: An image data storage unit for storing image data;
An image projection unit that projects a projection image based on the image data stored in the image data storage unit onto a projection object;
An imaging unit that captures a description image described in the projection object on which the projection image is projected, and acquires description image data of the description image;
An image data processing program readable by a computer of an image data processing apparatus comprising:
Based on the description image data acquired by the imaging unit, at least a part of the description range in which the description image is described is included in a projection range in the projection target on which the projection image is projected. A judging means for judging whether or not there is,
When the determination unit determines that at least a part of the description range is not included in the projection range, composite image data obtained by combining the projection image data corresponding to the projection image and the description image data is obtained. Data synthesizing means to generate;
Synthesized data output means for outputting the synthesized image data synthesized by the data synthesizing means;
The computer to function,
A specific position in the projection image projected on the projection object is a first projection reference position,
A specific position in the description image described in the projection object, and a position existing in the projection image projected on the projection object is a first description reference position;
A position corresponding to the first projection reference position in the image based on the composite image data is a second projection reference position;
A position corresponding to the first description reference position in an image based on the composite image data is a second description reference position;
When defined,
The data synthesizing means includes
The relative position of the second description reference position with respect to the second projection reference position is the same as the relative position of the first description reference position with respect to the first projection reference position, or the second description An image data processing program for generating the composite image data in a state where a relative position of a reference position with respect to the first described reference position can be recognized.

Images