JP6464599B2 - Image processing apparatus, image processing system, control method for image processing apparatus, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing system, a control method for the image processing apparatus, and a program.

  Since a projector can project an enlarged image, it is used in various situations such as a meeting in an office, various presentations, and an educational site where a presenter or the like presents the same image to a plurality of people. Also, depending on the presenter, a description may be added to the projection surface of the image by the projector with a pen, marker, or the like. In that case, the projection image projected on the projection surface may interfere with the explanation with a pen or marker.

  As a related technology, a projector is known that can specify an area for an input image using a cursor button on the projector main body or a remote control, and replace the color of the specified area with a background color. (For example, refer to Patent Document 1).

  In the technique disclosed in Patent Document 1, a menu screen is selected by controlling a projector by operating a cursor button such as an input operation unit of a projector or a remote controller, and an area to be processed is designated. For this reason, for example, a user who makes a presentation using a projector performs an operation for designating an area to be processed with respect to the projector while confirming a projection image, and the operability is not necessarily good. It was. As described above, it is difficult for a user who makes a presentation using an image processing apparatus (a projector or the like) that projects an image by a projection unit to instruct image processing on a predetermined area of the projected image. It was.

  An embodiment of the present invention has been made in view of the above problems, and an image processing apparatus that makes it easy to instruct image processing on a predetermined area of a projection image projected by a projection unit. The purpose is to provide.

In order to solve the above-described problem, an image processing apparatus according to an embodiment of the present invention is an image processing apparatus that projects a predetermined image on a projection plane by a projection unit, and captures a projection image projected on the projection plane. Determination means for determining a designated area designated for the projection image based on the captured image obtained by calculating, and a calculating means for calculating a position of the decided designated area in the predetermined image to be projected; Based on the calculated position, image processing means for performing predetermined image processing on an area corresponding to the specified designated area of the predetermined image to be projected, and a curve for designating the designated area are open Interpolating means for interpolating the curve into a closed curve when determined, and when the interpolating means determines that the curve specifying the specified region is open, at least one end point of the curve; and Song Calculating the area of the designated area when interpolating between the plurality of points with a straight line, and between the point on the curve where the area of the designated area is maximum and the at least one end point Complement with straight lines or curves .

  According to the embodiment of the present invention, it is possible to provide an image processing apparatus that makes it easy to instruct image processing on a predetermined region of a projection image projected by a projection unit.

It is a figure which shows an example of a structure of the image processing system which concerns on one Embodiment. It is a figure which shows the example of the projection image after replacing the inside of the designation | designated area | region which concerns on one Embodiment. It is a figure which shows the hardware structural example of the image projector which concerns on one Embodiment. It is a figure which shows the structural example of the computer which concerns on one Embodiment. It is a functional lineblock diagram of the image conversion part concerning a 1st embodiment. It is a figure for demonstrating the outline | summary of the process of the image conversion part which concerns on 1st Embodiment. It is a figure for demonstrating the process of the image comparison means and designated area determination means which concern on 1st Embodiment. It is a figure for demonstrating the process of the positional relationship calculation means which concerns on 1st Embodiment. It is a figure for demonstrating the process of the pixel replacement means which concerns on 1st Embodiment. It is a flowchart which shows the flow of the preview process which concerns on 1st Embodiment. It is a function structure part of the image conversion part which concerns on 2nd Embodiment. It is a figure for demonstrating the process of the area | region interpolation means which concerns on 2nd Embodiment. It is a figure which shows the structural example of the image processing system which concerns on 3rd Embodiment. It is a figure which shows the example of the light indicator which concerns on 3rd Embodiment. It is a functional block diagram of the image conversion part which concerns on 3rd Embodiment. 12 is a flowchart illustrating a flow of pixel replacement processing according to the third embodiment. It is a figure which shows the other structural example of the image processing system which concerns on one Embodiment.

  Embodiments of the present invention will be described below with reference to the accompanying drawings.

<System configuration>
FIG. 1 is a diagram illustrating an example of a configuration of an image processing system according to an embodiment. The image processing system 100 includes an image projection apparatus (projector) 101, a PC (Personal Computer) 102, a projection surface (screen) 103, and the like. The image processing system 100 can be suitably used in, for example, a conference system.

  For example, the image projection apparatus 101 acquires a predetermined image (for example, an image displayed on the PC 102) to be projected from an information terminal such as the PC 102 connected to the image projection apparatus 101, and the acquired image is converted into the projection lens 107. Is projected onto the projection plane 103. In addition, the image projection apparatus 101 according to the present embodiment includes a projection plane 103 and a camera 108 that captures a projection image 104 projected on the projection plane 103. The camera 108 may be built in the image projection apparatus 101 or may be provided outside the image projection apparatus 101. Note that the image projection apparatus 101 is an example of an image processing apparatus.

  When the user of the image processing system 100 has a region to be deleted in the projection image 104, for example, when a space is required to write a comment on the projection image 104, the user deletes it with the pointing tool 105 such as a pen or a penlight. A designated area 106 to be drawn is drawn on the projection plane 103. The projection plane 103 and the projection image 104 on which the designated area 106 is drawn are captured by the camera 108 and input to the image projection apparatus 101.

  The image projection apparatus 101 replaces the inside of the designated area 106 designated by the pointing tool 105 with, for example, a predetermined color (background color or the like) based on a captured image captured by the camera 108. As a result, a space for writing a comment or the like in the designated area 106 designated in the projection image 104 projected on the projection plane 103 is created. The process of replacing the inside of the designated area 106 with a predetermined color is an example of image processing. For example, the image projection apparatus 101 may perform image processing so that the information in the designated area 106 is not conspicuous by changing the color tone of the designated area 106 or changing the font or line color. good.

  Further, the user of the image processing system 100 can save the image acquired by the camera 108 in the PC 102 at an arbitrary timing. Since this image is saved with a different file name, the user can save as many images on the PC 102 as the storage capacity of the PC 102 allows.

  FIG. 2 is a diagram illustrating an example of a projected image after replacing the inside of the designated area according to an embodiment. In the example of FIG. 2, the pixel value inside the designated area 106 drawn in the projection image 104 is replaced with the background color. A user of the image processing system 100 can suitably add information such as a comment in the designated area 106 replaced with the background color. Further, the projected image 104 with the comment added can be captured by the camera 108 and stored in the PC 102.

  As described above, according to the image processing system 100 according to the present embodiment, a user who makes a presentation using the image projection apparatus 101 performs predetermined image processing (for example, background color) on the designated area 106 of the projection image 104. Can be easily instructed.

<Hardware configuration>
(Hardware configuration of image projector)
FIG. 3 is a hardware configuration diagram of the image projection apparatus according to the embodiment. The image projection apparatus 101 includes a CPU (Central Processing Unit) 301, a RAM (Random Access Memory) 302, a ROM (Read Only Memory) 303, an I / F (Interface) 304, an optical engine 305, and an internal bus 311. An image conversion unit 306 is included.

  The CPU 301 is an arithmetic device that controls the entire image projection apparatus 101 by executing a program stored in the ROM 303 or the like.

  A RAM 302 is a non-volatile memory used as a work area for the CPU 301. The RAM 302 stores various image data such as image data input from the PC 102, captured image data captured by the camera 108, and image data after the pixel values in the designated area 106 are replaced.

  The ROM 303 is a non-volatile memory that stores a program, and includes, for example, a flash ROM.

  The I / F 304 is a peripheral bus, a DMAC (Direct Memory Access Controller), a bus controller, or the like, and arbitrates the priority of data received by each interface connected to the I / F 304 and stores it in the RAM 302. Data is output to each interface connected to the I / F 304. In the example of FIG. 3, a network I / F 307, an external storage I / F 308, a video cable I / F 309, a camera I / F 310, and the like are connected to the I / F 304.

  The network I / F 307 is an interface for connecting the image projection apparatus 101 to a network by, for example, a wired / wireless LAN (Local Area Network) or the like and transmitting / receiving data. The external storage I / F 308 is an interface for connecting, for example, an external storage 312 of a USB (Universal Serial Bus) memory, a memory card, an external HDD (Hard Disk Drive), or an SSD (Solid State Drive). The video cable I / F 309 is an interface for connecting a video interface such as an RGB cable and an HDMI (registered trademark) (High-Definition Multimedia Interface) cable.

  The camera I / F 310 is an interface for connecting the camera 108. The camera 108 is an imaging unit for capturing the projection image 104 projected on the projection plane 103. The camera 108 may be built in the image projection apparatus 101 or may be externally attached to the image projection apparatus 101. When the camera 108 is externally attached, a captured image captured by the camera 108 may be input via a video cable I / F 309, a network I / F 307, or the like.

  The optical engine 305 includes, for example, a lamp, a light source for projection such as an LED (Light Emitting Diode), a display element such as a DMD (Digital Mirror Device) and an LCD (Liquid Crystal Display), a control unit, and the like. The image output from the optical engine 305 is projected as a projection image 104 on the projection surface 103 via the projection lens 107. As the projection lens 107, a lens in accordance with specifications such as a focal length, brightness, and angle of view is used in addition to a fixed focus lens or a zoom lens according to the specifications of the projector.

  The image conversion unit 306 determines the designated area 106 based on the captured images of the projection plane 103 and the projected image 104 captured by the camera 108, and performs predetermined image processing on the designated area 106 of the projected image 104. The image conversion unit 306 includes, for example, a dedicated circuit such as an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA), and is configured to execute a predetermined program. Note that the function of the image conversion unit 306 may be realized by, for example, a program operating on the CPU 301 or the like.

(PC hardware configuration)
The PC 102 is an information processing apparatus having a general computer configuration.

  FIG. 4 is a diagram illustrating a hardware configuration example of a computer according to an embodiment. The computer 400 includes, for example, a CPU 401, a RAM 402, a ROM 403, a storage unit 404, an input device 405, a display device 406, an external I / F 407, a communication I / F 408, a bus 409, and the like.

  The CPU 401 is an arithmetic device that reads programs and data from the ROM 403, the storage unit 404, and the like onto the RAM 402 and executes processing. A RAM 402 is a non-volatile memory used as a work area for the CPU 401. The ROM 403 is, for example, a non-volatile memory that stores a basic input / output system (BIOS) executed when the computer 400 is started up, various settings, and the like, and is configured by, for example, a flash ROM.

  The storage unit 404 is a non-volatile storage device that stores, for example, an OS (Operating System), application programs, various data, and the like, and includes, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like. .

  The input device 405 is realized by, for example, a keyboard, a mouse, a touch panel, and the like, and is used for inputting each operation signal to the computer 400. The display device 406 is realized by, for example, an LCD (Liquid Crystal Display) display or the like, and displays a processing result or the like by the computer 400.

  The external I / F 407 is an interface with an external device. The external devices include, for example, the image projection device 101, an external camera, various recording media, and the like.

  The communication I / F 408 is an interface that connects the computer 400 to a network. Thereby, the computer 400 can transmit and receive data via the communication I / F 408. A bus 409 is connected to each of the above components and transmits an address signal, a data signal, various control signals, and the like.

<Functional configuration>
[First Embodiment]
FIG. 5 is a functional configuration diagram of the image conversion unit according to the first embodiment. The image conversion unit 306 includes an image comparison unit 503, a designated area determination unit 504, a positional relationship calculation unit 505, a pixel replacement unit 506, and the like.

  The imaging unit 501 is a unit that captures the projection plane 103 and a projection image projected on the projection plane 103, and is realized by, for example, the camera 108 in FIG.

  The image comparison unit 503 extracts a difference between at least two captured images captured by the imaging unit 501. The image comparison unit 503 may be included in the designated area determination unit 504.

  The designated area determination unit (determination unit) 504 determines the designated area 106 designated for the projection image based on the captured image obtained by imaging the projection image projected on the projection plane 103 by the imaging unit 501. . In the present embodiment, the designated area determination unit 504 determines that the designated area 106 has been designated based on the difference between at least two captured images extracted by the image comparison unit 503, and designates the designated designated area 106 or the like. to decide.

  The positional relationship calculation unit (calculation unit) 505 calculates the position of the designated area 106 determined by the specified area determination unit 504 in the image projected by the projection unit 502. For example, the positional relationship calculation unit 505 calculates the positional relationship between the designated area 106 determined by the specified area determination unit 504 and the image projected by the projection unit 502.

  The pixel replacement unit 506 replaces the pixel of the image to be projected corresponding to the designated area 106 based on the position calculated by the positional relationship calculation unit 505. For example, the pixel replacement unit 506 sets a pixel value of a pixel corresponding to the specified area 106 among the pixels of the image to be projected as a predetermined value, for example, a background color, a specified color, or the like. Replace with. Note that the process of replacing a pixel value with a predetermined value is an example of a predetermined image process, and the pixel replacement unit 506 is an example of an image processing unit.

  The image comparison unit 503, the designated area determination unit 504, the positional relationship calculation unit 505, the pixel replacement unit 506, etc. are, for example, a program that operates in the image conversion unit 306 in FIG. Etc. Alternatively, the image comparison unit 503, the designated area determination unit 504, the positional relationship calculation unit 505, the pixel replacement unit 506, and the like may be realized by a program that operates on the CPU 301 in FIG.

  When the pixel replacement unit 506 replaces the pixel in the designated area 106 of the image to be projected, the projecting unit 502 projects the image to be projected onto the projection plane 103 after the pixel is replaced. The projection unit 502 is realized by, for example, the optical engine 305 and the projection lens 107 shown in FIG.

<Process flow>
(Outline of processing)
FIG. 6 is a diagram for explaining the outline of the processing of the image conversion unit according to the first embodiment.

  Here, the image projected by the projection unit 502 is, for example, image data input from the PC 102 and is stored in a predetermined area of the RAM 302 of the image projection apparatus 101. In the present embodiment, the designated area 106 is designated by a closed curve. The image comparison unit 503 will be described as being included in the designated area determination unit 504.

  In step S <b> 61, the designated area determination unit 504 extracts area data 604 on which only a curve designating the designated area 106 is drawn by calculating a difference between at least two captured images captured by the imaging unit 501.

  In step S <b> 62, the positional relationship calculation unit 505 calculates the positional relationship between the image data 603 projected by the projection unit 502 and the region data 604 extracted by the designated region determination unit 504.

  In step S63, the pixel replacement unit 506 replaces the pixel value of the pixel corresponding to the designated area 106 of the image data 603 to be projected with a predetermined value based on the positional relationship calculated by the positional relationship calculation unit 505. Further, the pixel replacement unit 506 stores the replaced image data 605 in a predetermined area of the RAM 302 as an image projected by the projection unit 502.

  In step S64, the projection unit 502 projects the replaced image data 605 stored in the RAM 302 onto the projection plane as a new image to be projected.

  Next, specific examples of processing contents of the image comparison unit 503, the designated area determination unit 504, the positional relationship calculation unit 505, and the pixel replacement unit 506 will be described.

(Processing of image comparison means and designated area determination means)
FIG. 7 is a diagram for explaining processing of the image comparison unit and the designated area determination unit according to the first embodiment.

  The image comparison means 503 calculates a difference between pixel values at the same position for a plurality of pixels of the first image 601 and the second image 602, preferably all of the pixels, and sets the difference to a predetermined threshold value. Extract more pixels.

  As shown in FIG. 7, when the first image 601 is an image before the designated area 106 is drawn and the second image 602 is an image after the designated area 106 is drawn, the designated area determining unit 504 includes an image comparing unit. From the extraction result 503, the area data 604 indicating the designated area 106 can be easily determined.

  If there is no pixel in which the difference between the pixel values at the same position in the first image 601 and the second image 602 exceeds a predetermined threshold value, the designated area determination unit 504 does not designate the designated area 106. It can be judged.

  On the other hand, when the pixel extracted by the image comparison unit 503 is not a closed curve, for example, when the second image 602 is captured in the middle of drawing the designated region 106, the designated region determination unit 504, for example, The second image 602 is updated with the captured image after a predetermined time has elapsed. In addition, the image comparison unit 503 recalculates the difference between the pixel values at the same position in the first image 601 and the updated second image 602. For example, when such processing is repeated and the area data 604 indicating the specified area 106 becomes a closed curve, the specified area determining unit 504 determines that the specified area 106 has been specified, and indicates the area indicating the specified area 106. The data 604 is determined (acquired).

  As another example, the designated area determination unit 504 determines that the designated area 106 has been designated based on the difference between the pixel values at the same position of three or more captured images captured at a predetermined time interval, The area data 604 indicating the designated area 106 may be acquired.

  In addition, when the designated area 106 is drawn with a penlight or the like having a light emitting unit, an image obtained by combining a plurality of images continuously captured in a short time such as a moving image, or by increasing the exposure time of the camera 108 The area data 604 can be determined from an image or the like obtained by capturing the locus.

  The region data 604 determined (acquired) by the specified region determination unit 504 by the above processing includes a closed curve that specifies the specified region 106.

(Processing of positional relationship calculation means)
FIG. 8 is a diagram for explaining the processing of the positional relationship calculation unit according to the first embodiment. The positional relationship calculation unit 505 identifies the position in the image to be projected where the area data 604 indicating the specified area 106 determined by the specified area determining unit 504 is drawn.

  As shown in FIG. 8A, the image 801 captured by the imaging unit 501 includes a background portion 803 such as the projection plane 103 in addition to the projection image 802 by the projection unit 502. Therefore, the positional relationship calculation unit 505 first cuts out the background portion 803 from the captured image 801 and extracts only the projection image 802.

  In order to cut out the background portion 803 from the captured image 801, the boundary between the projection image 802 and the background portion 803 is determined. For example, when the background portion 803 is the projection surface 103 such as a whiteboard and has the same color, and the projection image 802 is a rectangle, the projection image 802 and the background portion 803 have edges in the horizontal and vertical directions. Exists.

  Therefore, the horizontal and vertical edge amounts for each pixel are acquired. The edge amount indicates an output value after the edge detection filter is used for the target pixel, and is calculated for each pixel. There are various types of edge detection filters, but the types are not limited here.

  After acquiring the edge amount of each pixel, the sum of the edge amounts in the horizontal direction is calculated for all the rows, and a predetermined number of rows having a large sum value are stored from the top. Similarly, the sum of the edge amounts in the vertical direction is calculated for all the columns, and a predetermined number of columns having a large sum value are stored from the top.

  Among the predetermined number of stored lines, two lines that are most open are selected, and the selected two lines are set as upper and lower boundaries between the projection image 802 and the background portion 803. Similarly, two rows that are most open between the columns are selected from the predetermined number of stored columns, and the selected two rows are used as the left and right boundaries between the projection image 802 and the background portion 803.

  By extracting an image along the boundary, a projected image 802 can be extracted from the captured image 801.

  Next, since the projection image 802 captured by the imaging unit 501 and the original image projected by the projection unit 502 are different in size, a ratio is calculated in order to know the relative positional relationship.

  As shown in FIG. 8B, the width of the projected image 802 extracted from the captured image 801 is W, the height is H, and the coordinates of each pixel of the area data 604 indicating the designated area 106 are (x, y). In this case, the horizontal ratio is calculated as x / W, and the vertical ratio is calculated as y / H. The positional relationship calculation unit 505 calculates a ratio for each pixel of the area data 604 indicating the designated area 106.

  The horizontal image size (width) of the image data 603 projected by the projection unit 502 stored in a predetermined area of the RAM 302 of the image projection apparatus 101 is w, and the vertical image size (height) is h. To do. In this case, the coordinates of each pixel of the area data 604 corresponding to the designated area 106 in the image data 603 projected by the projection unit 502 are represented by (xw / W, yh / H). Thereby, the positional relationship calculation unit 505 can specify the position of each pixel of the area data 604 indicating the designated area 106 included in the captured image 801 in the image projected by the imaging unit 501.

(Processing of pixel replacement means)
FIG. 9 is a diagram for explaining processing of the pixel replacement unit according to the first embodiment. The pixel replacement unit 506 replaces the pixel value of the pixel corresponding to the designated area 106 in the image data 603 (image before substitution) projected by the projection unit 502 with a predetermined value.

  As an example of the predetermined value, for example, as shown in FIG. 9B, the pixels inside the area 901 are replaced with the background color of the image data 603 to be projected. In this case, for example, the frequency of the pixel value existing at the boundary of the region 901 is calculated, and the pixel value of the pixel in the region 901 is replaced with the pixel value having the highest frequency.

  As another example of the predetermined value, for example, as shown in FIG. 9C, the pixels inside the region 901 are replaced with an arbitrary color. In this case, the pixel value of the pixel in the area 901 is replaced with a pixel value of a preset designated color.

  Note that the processing of the image comparison unit 503, the designated area determination unit 504, the positional relationship calculation unit 505, and the pixel replacement unit 506 is an example, and does not limit the scope of the present invention. The image comparison unit 503, the designated area determination unit 504, the positional relationship calculation unit 505, and the pixel replacement unit 506 may realize the functions illustrated in FIG. 6 by other processing.

(Preview process)
FIG. 10 is a flowchart showing the flow of the preview process according to the first embodiment.

  Preferably, the image projection apparatus 101 desirably has a function of displaying a preview screen before confirming replacement of a pixel corresponding to the designated area 106 drawn by the user with a designated color or the like. As a result, it is possible to reduce the image processing (replacement) performed on a region different from the user's intention.

  In step S1001 of FIG. 10, for example, the pixels in the designated area 106 are replaced with a predetermined color based on the designated area 106 designated on the projection plane 103 by the process shown in FIG.

  In step S1002, the image in which the pixel is replaced in step S1001 is displayed (projected) as a preview. At this time, the image before pixel replacement is stored on a memory (such as the RAM 302).

  In step S1003, the user is prompted to select, for example, by displaying a screen for selecting whether or not to replace the pixel as previewed, and the process branches according to the user input. The user performs an operation indicating affirmation (for example, “OK” is selected) when performing replacement according to the preview display, and performs an operation indicating “No” (for example, selecting “cancel”) when different from the intended region.

  If a negative operation is performed in step S1003, pixel replacement is canceled in step S1004, the image before replacement of the pixels accumulated in the memory is projected, and the region is rewritten. After rewriting the area, the process returns to step S1001 to continue the processing.

  On the other hand, if an affirmative operation is performed in step S1003, the preview displayed image is projected in step S1005.

  Through the above processing, pixel replacement that is different from the user's intention can be easily restored, for example, when the user mistakenly designates the region, or when the image processing system 100 misrecognizes the region. It becomes like this.

[Second Embodiment]
In the present embodiment, an interpolation process will be described in the case where a curve for designating the designated area 106 by the user is not closed after drawing.

  FIG. 11 is a functional configuration diagram of an image conversion unit according to the second embodiment. The image conversion unit 306 according to the present embodiment has a region interpolation unit 1101 in addition to the configuration of the image conversion unit 306 according to the first embodiment shown in FIG. Since the configuration other than the region interpolation unit 1101 is the same as that in the first embodiment, only the difference will be described here.

  FIG. 12 is a diagram for explaining the operation of the region interpolation means according to the second embodiment. The curve 1201 is drawn with the intention that a user (eg, a presenter) of the image processing system 100 designates the designated area 106, but the curve 1201 is not closed because the drawing is not accurate. It is assumed that drawing has been completed in the state.

  Note that whether the curve 1201 is being drawn or whether drawing has ended can be determined, for example, based on whether the curve 1201 has changed after a predetermined time has elapsed. For example, when it is determined that the drawn curve 1201 is open for a predetermined time or more, the region interpolation unit 1101 interpolates the curve 1201 into a closed curve.

  A dotted line a in FIG. 12 shows an example in the case where the start point and end point of the curve 1201 are complemented with a straight line. The line that complements the start point and end point of the curve 1201 may be a curve.

  A dotted line b in FIG. 12 shows an example in the case of complementing with a straight line from the end point of the curve 1201 so that the area of the designated region 106 is maximized.

  For example, the area interpolation unit 1101 calculates the area in the area when a plurality of straight lines are drawn at a predetermined angle (for example, in units of 1 degree) starting from each of the two end points of the curve 1201 and interpolated with the respective straight lines. To do. In addition, it is a preferable example to draw a plurality of straight lines at a predetermined angle. In short, a straight line is drawn between two end points of the curve 1201 and a plurality of points on the curve 1201, and each straight line is a curve. What is necessary is just to calculate the area in the area | region at the time of interpolating 1201 to a closed curve.

  For example, the calculated area value is stored in the memory, compared with the value stored in the memory when the next straight line is drawn, and if the calculated value is larger, the value stored in the memory is updated. To do. Thereby, the point on the curve 1201 where the area in the region is maximum can be specified.

  The region interpolation unit 1101 complements a point on the curve 1201 when the area of the region is maximum and the end point of the curve 1201 with a straight line or a curve.

  A dotted line (c) in FIG. 12 shows an example in which the curve 1201 is interpolated from the end point of the curve 1201 by a straight line extending the curve 1201. For example, the straight line is created in a direction passing through the end point of the curve 1201 and a point in the vicinity of the end point (for example, a point adjacent to the end point).

  The above interpolation method is an example, and an open curve may be interpolated into a closed curve by a method other than the above.

  By the above processing, even when the curve for designating the designated area 106 is not closed after the drawing is completed, the designated area 106 can be identified by interpolating the curve for designating the designated area 106. Become.

[Third Embodiment]
(System configuration)
FIG. 13 is a diagram illustrating a configuration example of an image processing system according to the third embodiment. In the image processing system 100 shown in FIG. 1, when there is an area to be deleted in the projection image 104, it is assumed that a designated area 106 (closed curve) to be deleted is drawn on the projection plane 103 with a pointing tool 105 such as a pen or pen light. Went. However, the designation of the designated area 106 with a closed curve is merely an example, and the area may be designated on the projection plane 103 by another method.

  In the present embodiment, an example will be described in which a designated area is designated by an optical indicator 1301 such as a laser pointer instead of the indicator 105 such as a pen or a penlight.

  In FIG. 13, when the user displays a light spot on the projection plane using the light output from the light indicator 1301, the image projection apparatus 101 displays based on the captured image captured by the camera 108. A cursor 1302 is displayed around the light spot. As a result, the user can display and move the cursor 1302 to an arbitrary location on the projection plane 103 using the light indicator 1301. Preferably, the size of the cursor 1302 can be changed by, for example, a user operation or setting of the image projection apparatus 101.

  In addition, when a predetermined operation (for example, pressing of a replace button) is performed by the user, the image projection apparatus 101 determines that the area in the cursor 1302 is a specified area, and sets the pixels in the specified area to a predetermined color (pixel value). ). Further, when another predetermined operation (for example, pressing of the recovery button) is performed by the user, the image projection apparatus 101 can return the replaced pixel in the cursor 1302 to the pixel value before replacement.

(Configuration of light indicator)
FIG. 14 is a diagram illustrating an example of a light indicator according to the third embodiment. The light indicator 1301 includes, for example, a light source unit 1401, a light irradiation button 1402, a dial 1403, a replacement button 1404, a recovery button 1405, and the like.

  The light source unit 1401 is a light source for displaying a light spot, and is realized by, for example, a semiconductor light source such as a semiconductor laser or an LED (Light Emitting Diode). The light irradiation button 1402 is a button for controlling turning on / off of the light source unit 1401. A dial 1403 is used to change the size of the cursor 1302. A replace button 1404 is a button for instructing replacement of the pixel in the cursor 1302. A recovery button 1405 is a button for instructing the recovery of the replaced pixel value in the cursor 1302.

  Note that at least a part of the operation performed on the light indicator 1301 is notified to the image projection apparatus 101, the PC 102, and the like using, for example, infrared communication, short-range wireless communication, wireless LAN, or wired cable. Shall.

(Functional configuration)
FIG. 15 is a functional configuration diagram of an image conversion unit according to the third embodiment. The image conversion unit 306 according to the present embodiment includes a light instruction detection unit 1501 and a cursor display unit 1502 instead of the image comparison unit 503 of the image conversion unit 306 according to the first embodiment shown in FIG. Further, a pixel replacement / recovery unit 1505 is provided instead of the pixel replacement unit 506 of the image conversion unit 306 according to the first embodiment shown in FIG.

  The light instruction detection unit 1501 detects the light spot displayed on the projection surface 103 by the light indicator 1301 based on the projected image captured by the imaging unit 501 and the captured image obtained by capturing the projection image 104. Further, the position of the detected light spot with respect to the image projected by the projection unit 502 is calculated, for example, in the same manner as the processing by the positional relationship calculation unit 505 of the first embodiment.

  The cursor display unit 1502 displays a predetermined cursor 1302 around the position of the light spot calculated by the light instruction detection unit 1501 of the image. In the example of FIG. 13, the cursor 1302 has a rectangular shape, but the cursor 1302 may have a shape other than a rectangle.

  For example, the designated area determination unit 1503 detects that the replace button 1404, the recovery button 1405, or the like of the light indicator 1301 has been pressed, and determines (specified) the designated area within the range of the cursor 1302 at that time. To do.

  The positional relationship calculation unit 1504 calculates the position of the designated area determined by the designated area determination unit 1503 in the image projected by the projection unit 502. The positional relationship calculation unit 1504 is calculated in the same manner as the processing by the positional relationship calculation unit 505 of the first embodiment, for example.

  When the replace button 1404 of the light indicator 1301 is pressed, the pixel replacement / recovery unit 1505 replaces the pixel in the region corresponding to the cursor 1302 of the image projected by the projection unit 502 with a predetermined pixel value. Further, when the recovery button 1405 of the light indicator 1301 is pressed, the pixel replacement / recovery unit 1505 recovers the replaced pixel in the area corresponding to the cursor 1302 of the image projected by the projection unit 502 to the original pixel value. Let The pixel replacement / recovery unit 1505 is an example of an image processing unit.

(Process flow)
FIG. 16 is a flowchart illustrating the flow of pixel replacement processing according to the third embodiment.

  In step S1601, the image projection apparatus 101 determines whether or not the projection surface 103 is irradiated with light by the light indicator 1301, and when the light is irradiated, the process proceeds to step S1602. On the other hand, when light is not irradiated, the process of step S1601 is repeated.

  In step S <b> 1602, the image projection apparatus 101 displays a cursor 1302 around the light emitted by the light indicator 1301. Note that the cursor 1302 moves following the light emitted from the light indicator 1301.

  In step S1603, the image projection apparatus 101 determines whether the button 1 is pressed. The button 1 is a button for the user to instruct pixel replacement, and corresponds to the replacement button 1404 of the light indicator 1301, for example.

  If button 1 is pressed in step S1603, the pixel value in cursor 1302 is replaced in step S1604. This operation is effective while the button 1 is being pressed. When the cursor is moved while the button 1 is being pressed, the pixels in the button 1 are replaced. On the other hand, if button 1 is not pressed in step S1603, the process returns to step S1602 and the process is repeated.

  In step S1605, the image projection apparatus 101 determines whether the button 2 is pressed. The button 2 is a button for the user to instruct recovery of the replaced pixel, and corresponds to the recovery button 1405 of the light indicator 1301, for example.

  If button 2 is pressed in step S1605, the replaced pixel value in cursor 1302 is returned to the pixel value before replacement in step S1606. This operation is effective while the button 2 is pressed. When the cursor is moved while the button 2 is pressed, the replaced pixel in the button 2 returns to the pixel value before replacement.

  As a preferred example, when the button 2 is pressed twice quickly, all the pixels replaced by the button 1 return to the pixel values before the replacement.

  With the above processing, the user of the image processing system 100 can replace at least a part of information of the projection image 104 projected on the projection plane 103 in real time and can restore the information before the replacement. .

[Other system configurations]
The system configuration of the first to third embodiments is an example, and the image processing system 100 can have various system configurations.

  FIG. 17 is a diagram illustrating another configuration example of the image processing system according to the embodiment. For example, in the first to third embodiments, the image projection apparatus 101 has been described as including the image conversion unit 306. However, the image conversion unit 306 may be included in the PC 102. In this case, each unit of the image conversion unit 306 is realized by, for example, a program operating on the CPU 401 in FIG.

  A configuration example of the image processing system 100 when the image conversion unit 306 is included in the PC 102 is shown in FIG. In this case, for example, the imaging unit 501 in FIG. 5 is realized by the camera 108, and the projection unit 502 is realized by the image projection apparatus 101.

  The PC 102 may be an information terminal other than the PC 102 such as a tablet terminal. An information terminal such as the PC 102 is another example of an image processing apparatus.

  Further, the camera 108 may be a camera built in an information terminal such as the PC 102, or may be a plurality of cameras. By providing a plurality of cameras, it is possible to reduce blind spots when a user enters between the camera 108 and the projection plane 103.

  Further, as shown in FIG. 17B, the image processing system 100 does not necessarily have an information terminal such as the PC. In this case, the image to be projected may be acquired from the network using the network I / F 307 in FIG. 3 or acquired from the external storage 312 using the external storage I / F 308 in FIG. It may be.

100, 1300 Image processing system 101 Image projection apparatus (an example of an image processing apparatus)
102 PC (another example of an image processing apparatus)
DESCRIPTION OF SYMBOLS 103 Projection surface 104 Projection image 106 Designated area 501 Imaging means 502 Projection means 503 Image comparison means 504 Designated area determination means (determination means)
505 Positional relationship calculation means (calculation means)
506 Pixel replacement means (an example of image processing means)
1101 Image interpolation means (interpolation means)
1505 Image replacement / recovery means (another example of image processing means)

JP 2008-233742 A

Claims (12)

An image processing apparatus that projects a predetermined image on a projection surface by a projection unit,
Determination means for determining a designated area designated for the projection image based on a captured image obtained by imaging the projection image projected on the projection plane;
Calculating means for calculating a position of the determined designated area in the predetermined image to be projected;
Image processing means for performing predetermined image processing on an area corresponding to the specified area of the predetermined image to be projected based on the calculated position;
When it is determined that a curve that specifies the specified region is open, an interpolation unit that interpolates the curve into a closed curve;
I have a,
The interpolation means includes
When it is determined that the curve that specifies the specified area is open, the area of the specified area is calculated when a straight line is interpolated between at least one end point of the curve and a plurality of points on the curve. And
An image processing apparatus that complements a point on the curve where the area of the designated region is maximum and the at least one end point with a straight line or a curve . An image processing apparatus that projects a predetermined image on a projection surface by a projection unit,
Determination means for determining a designated area designated for the projection image based on a captured image obtained by imaging the projection image projected on the projection plane;
Calculating means for calculating a position of the determined designated area in the predetermined image to be projected;
Image processing means for performing predetermined image processing on an area corresponding to the specified area of the predetermined image to be projected based on the calculated position ;
Have,
The image processing means includes
Detecting a background color of the predetermined image to be projected;
An image processing apparatus that replaces a pixel corresponding to the designated area of the predetermined image to be projected with the detected background color . Even without least it includes an image comparison means to extract the difference between two of the captured image,
The determination means includes
The image processing apparatus according to claim 1, wherein the designated area is determined based on the extracted difference. The image comparison means includes
Calculating a difference between pixel values at the same position for a plurality of pixels of the at least two captured images;
The difference between the calculated pixel values pixels that exceeds the predetermined threshold is, determines that the pixels specifying the specified area, the image processing apparatus according to claim 3. Before SL designated region is designated by the pointing device having a light emitting portion,
The determination means includes
5. The specified designated area is determined based on the plurality of continuously captured images or the captured image obtained by capturing a locus of the light emitting unit. 6. the image processing apparatus. When detecting the light that has been predetermined for the projection plane based on the previous SL captured image has the cursor display means to display the cursor around the position corresponding to the detected light of the projected image,
The determination means includes
The image processing apparatus according to claim 1 , wherein when a predetermined operation is received, the range of the displayed cursor is determined as the designated designated area. Projection means for projecting a predetermined image on the projection surface;
Imaging means for imaging a projected image projected on the projection plane;
A determination unit that determines a specified area specified for the projection image based on a captured image captured by the imaging unit;
Calculating means for calculating a position of the determined designated area in the predetermined image to be projected;
Image processing means for performing predetermined image processing on an area corresponding to the specified area of the predetermined image to be projected based on the calculated position;
When it is determined that a curve that specifies the specified region is open, an interpolation unit that interpolates the curve into a closed curve;
Have,
The interpolation means includes
When it is determined that the curve that specifies the specified area is open, the area of the specified area is calculated when a straight line is interpolated between at least one end point of the curve and a plurality of points on the curve. And
An image processing system that complements a point on the curve where the area of the designated area is maximum and the at least one end point with a straight line or a curve . Projection means for projecting a predetermined image on the projection surface;
Imaging means for imaging a projected image projected on the projection plane;
A determination unit that determines a specified area specified for the projection image based on a captured image captured by the imaging unit;
Calculating means for calculating a position of the determined designated area in the predetermined image to be projected;
Image processing means for performing predetermined image processing on an area corresponding to the specified area of the predetermined image to be projected based on the calculated position;
  Have
The image processing means includes
Detecting a background color of the predetermined image to be projected;
An image processing system that replaces a pixel corresponding to the designated area of the predetermined image to be projected with the detected background color. An image processing apparatus that projects a predetermined image on a projection surface by a projection unit,
Determination means for determining a designated area designated for the projection image based on a captured image obtained by imaging the projection image projected on the projection plane;
Calculating means for calculating a position of the determined designated area in the predetermined image to be projected;
Image processing means for performing predetermined image processing on an area corresponding to the specified area of the predetermined image to be projected based on the calculated position;
When it is determined that a curve that specifies the specified region is open, an interpolation unit that interpolates the curve into a closed curve;
And then allowed to function,
The interpolation means includes
When it is determined that the curve that specifies the specified area is open, the area of the specified area is calculated when a straight line is interpolated between at least one end point of the curve and a plurality of points on the curve. And
The program which complements between the point on this curve where the area of the said designation | designated area | region becomes the maximum, and the said at least 1 end point with a straight line or a curve . An image processing apparatus that projects a predetermined image on a projection surface by a projection unit,
Determination means for determining a designated area designated for the projection image based on a captured image obtained by imaging the projection image projected on the projection plane;
Calculating means for calculating a position of the determined designated area in the predetermined image to be projected;
Image processing means for performing predetermined image processing on an area corresponding to the specified area of the predetermined image to be projected based on the calculated position;
Function as
The image processing means includes
Detecting a background color of the predetermined image to be projected;
A program for replacing a pixel corresponding to the designated area of the predetermined image to be projected with the detected background color . A control method for an image processing apparatus for projecting a predetermined image onto a projection surface by a projection means,
The image processing apparatus is
Determining a designated area designated for the projection image based on a captured image obtained by imaging the projection image projected on the projection plane;
Calculating a position of the determined designated area in the predetermined image to be projected;
Performing predetermined image processing on an area corresponding to the specified area of the predetermined image to be projected based on the calculated position ;
If it is determined that a curve specifying the specified region is open, complementing the curve with a closed curve;
Run
The complementing step includes
When it is determined that the curve that specifies the specified area is open, the area of the specified area is calculated when a straight line is interpolated between at least one end point of the curve and a plurality of points on the curve. And
A control method for an image processing apparatus , wherein a line or a curve is complemented between a point on the curve where the area of the designated area is maximum and the at least one end point . A control method for an image processing apparatus for projecting a predetermined image onto a projection surface by a projection means,
  The image processing apparatus is
Determining a designated area designated for the projection image based on a captured image obtained by imaging the projection image projected on the projection plane;
Calculating a position of the determined designated area in the predetermined image to be projected;
Performing predetermined image processing on an area corresponding to the specified area of the predetermined image to be projected based on the calculated position;
Run
The step of performing the image processing includes:
Detecting a background color of the predetermined image to be projected;
A control method of an image processing apparatus, wherein a pixel corresponding to the specified designated area of the predetermined image to be projected is replaced with the detected background color.

Images