JP6312488B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

2. Description of the Related Art In recent years, there has been proposed an image processing apparatus for projecting an image obtained by photographing a paper surface placed on a document table onto the document table by combining a conventional document camera and a projector. Such an image processing apparatus is mainly used for scanning a paper surface and viewing an image, and usually includes a paper surface detection function for detecting a range of the paper surface from a captured image. In addition, an input device that realizes touch input on a document table by detecting the position of a fingertip from an image obtained by camera shooting has been proposed, and is used in combination with the above-described image processing device.
However, such a paper surface detection function may cause an error. For example, a ruled line of a table printed inside a paper surface may be erroneously detected as a boundary of the paper surface range, or a straight line outside the paper surface range may be erroneously detected as a boundary. Therefore, in addition to the paper surface detection function, an area changing function for the user to change the detection area is required.
As a known method for realizing the area changing function, there is a method of selecting and moving a boundary of an area using a touch panel function attached to the display while browsing an image taken with a digital camera on a display attached to the camera. Patent Document 1 discloses a display control device that changes the size of an object displayed using a touch panel. In this display control apparatus, when the user touches and moves an area inside the object, the object is enlarged, reduced, or rotated based on the moving direction.

Japanese Patent No. 4908626

  In the method of changing the region boundary while viewing a captured image, the paper surface and the background may be assimilated depending on the shooting environment, and it may be difficult to visually recognize the paper surface boundary on the image. In this case, the region boundary cannot be changed correctly. In order to solve this, a method is conceivable in which a frame or the like indicating a detection region is superimposed and projected on the paper surface, and the user changes the region boundary while directly viewing and comparing the actual paper surface and the detection region. However, when the inside of the paper surface as an object is touched as in the technique disclosed in Patent Document 1 for this method, an operator such as a finger contacts the paper surface, and an intended operation cannot be performed.

  The present invention has been made in view of such problems, and an object thereof is to correct an object detection area in accordance with a natural touch operation by a user.

Therefore, the present invention provides an image processing apparatus, a photographing unit that photographs a photographing region including an object, a detection unit that detects an object region from a photographed image photographed by the photographing unit, and the above-described image in the photographed image. Projecting means for projecting an object image indicating the object area on the object based on the position of the object area, and a non-object other than the object area in the photographing area based on the position of the outer peripheral line of the object image A dividing unit that divides the region into a plurality of partial regions; an association unit that associates the plurality of partial regions with a partial outer periphery that constitutes the outer peripheral line; and an input position of an operator on the non-object region Target specifying means for specifying the partial outer periphery associated with the partial area as a correction target; By modifying the position of the positive subject, and a correction means for correcting the object image.

  According to the present invention, the detection area of an object can be corrected according to a natural touch operation by a user.

It is a figure which shows an image processing apparatus. It is a figure for demonstrating the utilization method of an image processing apparatus. It is a flowchart which shows a paper surface detection process. It is a flowchart which shows a paper surface image correction process. It is a figure for demonstrating a partial outer periphery and a partial area | region. It is a figure for demonstrating the other example of a partial outer periphery and a partial area | region. It is a figure for demonstrating a paper surface image correction process. It is a figure for demonstrating a paper surface image correction process. It is a figure which shows the example of a display of a side candidate. It is a figure for demonstrating the image processing apparatus concerning 2nd Embodiment. It is a flowchart which shows the paper surface detection process concerning 2nd Embodiment. It is a figure for demonstrating operation at the time of selecting a paper surface image. It is a figure for demonstrating the paper surface image correction process concerning 2nd Embodiment. It is a figure for demonstrating the image processing apparatus concerning 3rd Embodiment. It is a flowchart which shows the paper surface detection process concerning 3rd Embodiment. It is a figure for demonstrating the paper surface image correction process concerning 3rd Embodiment. It is a figure for demonstrating the example of a change of the image processing apparatus concerning 3rd Embodiment.

  Hereinafter, embodiments of the present invention will be described based on the drawings.

FIG. 1 is a diagram illustrating an image processing apparatus. The image processing apparatus includes a projection unit 101, a photographing unit 102, a UI unit 103, a control unit 104, a storage unit 105, and a communication unit 106. The projection unit 101 is a projector that projects image data onto a projection surface such as a screen. The imaging unit 102 is a digital camera or a video camera that captures an area including the projection table 201 and converts external light into image data. The UI unit 103 receives an instruction from the user to the image processing apparatus, and instructs the projection content and the projection method on the projection unit 101. The UI unit 103 is realized by, for example, buttons provided in the projection apparatus, a visible light camera for recognizing a gesture operation, an infrared camera, a pen-type device for performing a touch operation, or the like.
The control unit 104 provides image data to the projection unit 101 based on an instruction received from the UI unit 103, or performs processing on the image data obtained from the imaging unit 102. The control unit 104 is, for example, a CPU or a GPU. The storage unit 105 includes a memory and a hard disk that hold the image data obtained from the imaging unit 102 and the processing results performed by the control unit 104. The communication unit 106 communicates with the image processing apparatus and other apparatuses, a file server (not shown), an application server, and the like. Note that the functions and processes of the image processing apparatus to be described later are realized by the control unit 104 reading a program stored in the storage unit 105 and executing the program.

FIG. 2 is a diagram for explaining a method of using the image processing apparatus. As shown in FIG. 2A, a paper surface 210 is placed on the projection table 201. Then, an area including the paper surface 210 is imaged by the imaging unit 102. The control unit 104 detects a region of the paper surface 210, that is, a paper region, in the obtained captured image. Then, as illustrated in FIG. 2B, the projection unit 101 projects a paper surface image 220 indicating the detected paper surface area onto the paper surface 210. A paper surface image 220 shown in FIG. 2B is a quadrangular frame image indicating the boundary position of the paper surface region. However, the paper image only needs to indicate the boundary position of the detection area, and the specific image content is not limited to the embodiment.
By the way, as shown in FIG. 2B, due to erroneous detection of the paper surface area by the image processing apparatus, the actual edge of the paper surface 210 may not match the paper image 220. The image processing apparatus according to the present embodiment corrects the paper surface image 220 so that the paper surface image 220 matches the paper surface 210 in response to a touch operation by the user.
Note that the page according to the present embodiment is an example of an object, and the page area and the page image are examples of an object area and an object image, respectively.

FIG. 3 is a flowchart illustrating a paper surface detection process performed by the image processing apparatus. In step S <b> 301, the control unit 104 instructs the photographing unit 102 to perform photographing. In response to this, the photographing unit 102 photographs the projection table 201 on which a paper surface is placed as a photographing region (photographing process). In step S <b> 302, the control unit 104 detects a paper area from the captured image captured by the capturing unit 102 (detection process).
Next, in S303, the control unit 104 specifies the projection position of the paper image based on the position of the paper region in the captured image. Then, the control unit 104 instructs the projection unit 101 to project a paper image indicating the paper region onto the paper region. Here, the process of S303 is an example of a projection control process that instructs to project a paper image onto a paper surface based on the position of the paper region. In response to this, the projection unit 101 projects a paper image onto a paper region (on an object) (projection processing).

In step S <b> 304, the control unit 104 determines whether the user has made a touch input to the projection table 201. The user confirms the paper image projected on the projection table 201. If the paper image does not match the actual paper surface, it is determined that there is an error in detecting the paper region, and touch input is performed. It should be noted that the area to be touched by the user is performed on an area other than the paper area, that is, a non-paper area. This is because, when an operator such as a user's finger is touched on the paper, there is a disadvantage that the paper also moves due to the subsequent movement of the operator. Here, the non-paper area is an example of a non-object area that is an area other than the object area.
The control unit 104 detects the presence or absence of touch input based on images captured by a plurality of cameras (not shown). As another example, the control unit 104 may detect the presence / absence of a touch input based on a detection result by a distance measuring sensor (not shown). Specifically, the control unit 104 measures the distance between the tip of the operator and the projection table 201, and determines that there is a touch input when the distance is equal to or less than a threshold value. As another example, the control unit 104 may determine the presence or absence of touch input based on signals from a switch, an acceleration sensor, or the like that is attached to the user's finger and detects contact. The coordinates of the input position of the touch input can be obtained by performing skin color detection from the captured image and performing fingertip detection based on the shape of the skin color region. As another example, the control unit 104 determines the presence or absence of touch input based on a signal from a marker or LED that is provided at the tip of a fingertip or a pen-type device and that can be easily detected from an image. May be.

If the control unit 104 determines that there is a touch input (Yes in S304), the control unit 104 advances the process to S305. When the control unit 104 determines that there is no touch input (No in S304), the control unit 104 advances the process to S306. In step S305, the control unit 104 corrects the paper image. Here, the process of S305 is an example of a correction process for correcting the paper image, that is, the object image, based on the input position of the operation element on the non-paper region, that is, the non-object region.
Next, in step S <b> 306, the control unit 104 saves the paper image in the storage unit 105. If the paper image is corrected in S305, the control unit 104 stores the corrected paper image.
As another example, in step S <b> 306, the control unit 104 may store position information indicating the position of the paper area corresponding to the paper image instead of the paper image or together with the paper image. In this case, when the paper image is not corrected, the control unit 104 stores the detection result of the paper region detected in S302 in the storage unit 105 as position information. In addition, when the paper image is corrected, the control unit 104 stores, in the storage unit 105, position information indicating a value obtained by correcting the detection result in S302 according to the correction of the paper image.

FIG. 4 is a flowchart showing detailed processing in the paper surface image correction processing. In step S401, the control unit 104 divides the non-paper area into a plurality of partial areas (division processing). FIG. 5 is a diagram for explaining a partial region. The control unit 104 divides the non-paper area into eight partial areas 501a to 501d and 502a to 502d based on the outer peripheral line of the rectangular paper area. Then, the control unit 104 associates a part of the outer peripheral line of the paper image with each partial region (association process).
In the example shown in FIG. 5, the partial areas 501 a to 501 d are quadrangular areas each having one side of the outer peripheral line. Each of the partial areas 502a to 502d is a quadrangular area in which the vertex of the outer peripheral line is one vertex and two sides extending from the vertex are shared with the partial areas 501a to 501d.

The division position and the number of divisions of the outer peripheral line are not limited to the embodiment. As another example, as shown in FIG. 6A, the control unit 104 uses the two lines 601a and 601b that connect the center of the paper area and the centers of the vertical and horizontal sides as a boundary line. May be divided into four partial regions 602a to 602d.
As another example, as shown in FIG. 6B, the control unit 104 uses the two lines 611a and 611b connecting the center of the paper area and each of the two adjacent vertices as a boundary line. The paper area may be divided into four partial areas 612a to 612d.

Returning to FIG. 4, after the process of S401, in S402, the control unit 104 specifies a partial outer periphery associated with the partial area including the input position of the touch input as a correction target (target specifying process). In step S <b> 403, the control unit 104 confirms whether the touch point input by touch input has moved. Here, the movement of the touch point means that the coordinates of the touch point change while maintaining a state where an operator such as a user's fingertip or a pen-type device is in contact with the projection plane. When the movement of the touch point is detected (Yes in S403), the control unit 104 advances the process to S404. If the movement of the touch point is not detected (No in S403), the control unit 104 advances the process to S406.
In step S <b> 404, the control unit 104 specifies a movement amount and a movement direction from the start point to the end point with the input position where the touch input is first made as the start point and the input position of the touch input after movement as the end point. Here, the process of S404 is an example of a movement amount specifying process and a moving direction specifying process. Next, in S405, the control unit 104 specifies the corrected position of the correction target based on the movement amount and movement direction specified in S404, and the object image is displayed so that the correction target is displayed at the specified position. Is corrected (correction processing). The correction process will be described later with reference to FIGS.

In step S <b> 406, the control unit 104 checks whether there is a change from a touch state in which the operation element is in contact with the projection surface to a non-contact state (release) in which the operation element is not in contact with the projection surface. When detecting the change to the non-contact state (Yes in S406), the control unit 104 advances the process to S407. If the control unit 104 does not detect a change to the non-contact state (No in S406), the control unit 104 advances the process to S403, detects the movement of the touch point, and corrects the arrangement position of the correction target corresponding to the detection result. Continue.
In S407, when the control unit 104 receives an end instruction from the user (Yes in S407), the paper surface image correction process ends. If the control unit 104 has not received an end instruction (No in S407), the control unit 104 advances the process to S401. Note that the control unit 104 accepts an end instruction when the user performs an operation on the UI unit 103 in order to proceed to the next step such as saving a paper image.

7 and 8 are diagrams for specifically explaining the paper surface image correction processing. With reference to FIG. 7, a process when the vertex A is selected as a correction target in S <b> 402 will be described. FIG. 7A and FIG. 7B show the paper images 220 and 720 before and after the paper image correction processing, respectively. As shown in FIG. 7A, it is assumed that the user touches the partial area 502b on the paper image 220 before correction. In contrast, the control unit 104 specifies the vertex A associated with the partial area 502b as a correction target.
Then, as illustrated in FIG. 7B, it is assumed that the user moves the touch point from the position 701 to the position 702 obliquely downward to the right. In this case, the control unit 104 corrects the display position of the vertex A from the original position 711 to a position obliquely downward to the right. As a result, a corrected paper image 720 is obtained. As described above, when the partial outer periphery corresponding to the apex is to be corrected, the vertical and horizontal sizes of the paper surface image can be enlarged or reduced according to the movement of the touch point by the user.
For example, the control unit 104 may set the moving direction from the start point to the end point as the correction direction of the vertex A, and the distance from the start point to the end point as the correction amount of the vertex A. As another example, in order to improve the operability for the user, the control unit 104 moves the movement amount so as to be proportional to the size of the paper image 220, the moving speed of the touch point, the size of the partial area, and the like. From this, the correction amount may be determined.

Next, the processing when the side B is selected as a correction target in S402 will be described with reference to FIG. FIG. 8A and FIG. 8B show the paper images 220 and 820 before and after the paper image correction processing, respectively. As shown in FIG. 8A, it is assumed that the user touches the partial area 501b on the paper image 220 before correction. In response to this, the control unit 104 identifies the side B associated with the partial region 501b as a correction target.
Then, as shown in FIG. 8B, it is assumed that the user moves the touch position from the point 801 to the point 802 in the right direction of the paper image. In this case, the control unit 104 corrects the display position of the side B from the original position to the rightward position. As a result, a corrected paper image 820 is obtained. As described above, when the partial outer circumference corresponding to the side is to be corrected, only the vertical or horizontal size of the paper image can be enlarged or reduced according to the movement of the touch point by the user.

  Note that the control unit 104 may use the distance from the start point to the end point as the correction amount of the side B, for example. As another example, the control unit 104 moves the movement amount so as to be proportional to the size of the paper image 220, the moving speed of the touch point, the size of the partial area, and the like, as in the case where the correction target is the vertex. From this, the correction amount may be determined.

  As described above, the image processing apparatus according to the present embodiment can correct the detection area (paper area) of an object placed on the projection table in response to a user touch operation on an area other than the paper area. it can.

  A first modification of the image processing apparatus according to the first embodiment will be described. Assume that the control unit 104 specifies a vertex as a correction target in S402 of FIG. 4 and detects the movement of the touch point in S403. In this case, the control unit 104 checks whether there is a vertex candidate in the vicinity of the input position of the touch point after movement based on the captured image. If there is a vertex candidate, the control unit 104 corrects the display position of the outer peripheral portion to be corrected to the position of this vertex candidate. Note that the control unit 104 detects vertex candidates by using a known method such as a Harris operator or a SUSAN operator.

  Next, a second modification example will be described. Assume that the control unit 104 specifies an edge as a correction target in S402 of FIG. In this case, the control unit 104 checks whether or not a side candidate exists near the side to be corrected. And when a side candidate exists, the control part 104 may display a side candidate. FIG. 9 is a diagram illustrating a display example of edge candidates. In the example shown in FIG. 9, the side candidates 901a and 901b are displayed together in a state where the paper image 220 is displayed as shown in FIG. Further, when the movement of the touch point is detected, the control unit 104 may use the side candidate in the movement direction as the corrected display position. Note that the control unit 104 detects edge candidates by using a known method such as Hough transform, RANSAC, or principal component analysis for the edge pixel group.

  As a third modification, the image processing apparatus can associate each partial region with at least one of the movement direction and the movement amount of a part of the associated outer peripheral line. For example, in FIG. 8A, the control unit 104 determines the moving direction and moving amount of the side B based on the distance between the side B and the point 801. When the distance is equal to or greater than the threshold value, the control unit 104 sets the direction of the display position after the correction of the outer peripheral portion to the outside from the center of the paper area. The direction of the position is the direction toward the center. Further, the image processing apparatus may determine the vertex position or the edge candidate as the display position of the corrected outer peripheral portion as described in the first and second correction examples without determining the movement amount. Further, the display position of the corrected outer peripheral portion may be determined based on the difference between the movement amount and the threshold value.

(Second Embodiment)
FIG. 10 is a diagram for explaining an image processing apparatus according to the second embodiment. Here, the difference between the image processing apparatus according to the second embodiment and the image processing apparatus according to the first embodiment will be described. As shown in FIG. 10 (a), when one sheet 1000 is placed on the projection table 201, as shown in FIG. 10 (b), two or more sheet images 1011 and 1012 are erroneously detected. To do. In this case, the image processing apparatus according to the second embodiment integrates a plurality of detected paper area into one paper area in response to a user operation.

FIG. 11 is a flowchart illustrating a paper surface detection process performed by the image processing apparatus according to the second embodiment. In FIG. 11, the same processes as those described with reference to FIG. After projecting the paper image in S303, in S1101, the control unit 104 confirms the number of paper regions detected in S302. If the number of detected paper area is greater than 1 (Yes in S1101), the control unit 104 advances the process to S1102. If the number of detected paper area is 1 or less (No in S1101), the control unit 104 advances the process to S304.
In step S <b> 1102, the control unit 104 confirms whether a selection input of a paper image from the user has been received. As described above, when a plurality of paper areas are detected, a paper image corresponding to each paper area is displayed (projected). At this time, when the user wants to perform the correction process, it is necessary to first select a paper image to be corrected.

FIG. 12 is a diagram for explaining an operation performed when the user selects a paper image. As shown in FIG. 12A, the user touches an arbitrary point 1201. The control unit 104 of the image processing apparatus shifts the operation mode to the area selection mode when a certain period of time has passed while being touched. Then, the control unit 104 determines that a paper image existing near the input position of the touch input has been selected. A point 1201 shown in FIG. 12A is closer to the paper image 1012 than the paper image 1011. Therefore, the control unit 104 determines that the paper image 1012 has been selected. The control unit 104 further displays a marker 1202 for the selected paper image. Thereby, the user can easily grasp which paper image is being selected.
Furthermore, as shown in FIG. 12B, it is assumed that the user has moved the input position to a point 1203 while being touched. In this case, the control unit 104 selects the paper image 1011 as the paper image closest to the input position, and displays the marker 1204 on the paper image 1011. The control unit 104 corrects the display position of the marker by a correction amount determined based on the movement amount of the input position as the input position moves. As another example, the control unit 104 may set the position in the paper image existing in the moving direction of the input position as the corrected marker display position. Note that the control unit 104 only needs to display information that allows the user to identify the selected paper image, and the specific display content for that purpose is not limited to the embodiment. As another example, the control unit 104 may display the frame of the selected paper image by blinking, changing the color, changing the thickness, or the like.

Returning to FIG. 11, the control unit 104 proceeds to S <b> 304 when the selection input of the paper image is accepted (Yes in S <b> 1102). If the control unit 104 does not accept the selection of the paper image (No in S1102), the control unit 104 advances the process to S1103. If the control unit proceeds to S304 and detects a touch input, the control unit performs a correction process on the selected paper image in S305, and then advances the process to S1103.
As shown in FIG. 13A, it is assumed that the paper surface image 1011 is in a selected state. In this state, the area other than the paper image 1011 is divided into partial areas corresponding to the eight partial outer circumferences of the outer circumference of the paper image 1011.
Then, as illustrated in FIG. 13B, it is assumed that the user performs touch input at a position 1301 in the partial area, and the input position moves to the position 1302 below the paper image. In this case, as shown in FIG. 13B, the size of the paper image 1011 is corrected to the paper image 1303 in the vertical direction.

Returning to FIG. 11, in step S <b> 1103, the control unit 104 determines whether or not a plurality of sheet images correspond to the same sheet (determination process). In other words, the control unit 104 detects an overlap between the corrected sheet area and another sheet area (detection process). Then, the control unit 104 determines whether or not a plurality of overlapping sheet images correspond to the same sheet based on the overlap detection result. Specifically, the control unit 104 specifies an overlap area when an overlap is detected, and when the overlap area is equal to or larger than the area threshold, the plurality of overlapped sheet images correspond to the same sheet. to decide.
For example, in the example shown in FIGS. 13A and 13B, as shown in FIG. 13B, the corrected paper image 1303 includes the paper image 1012, and the overlapping area is the area. Assume that the threshold is exceeded. In this case, the control unit 104 determines that the paper image 1303 and the paper image 1012 correspond to the same paper surface.

  Note that the control unit 104 only has to determine whether or not a plurality of paper surface images correspond to the same paper surface based on the overlap detection result, and specific processing for that is limited to the embodiment. It is not a thing. For example, the control unit 104 may determine whether or not a plurality of paper surface images correspond to the same paper surface based only on the presence or absence of overlap. Further, the control unit 104 may make a determination based on the number of vertices and sides existing in the overlap, the vertex or the side at a position designated in advance, or the like.

Returning to FIG. 11, when the control unit 104 determines in S1103 that a plurality of paper surface images correspond to the same paper surface (Yes in S1103), the process proceeds to S1104. On the other hand, if the control unit 104 determines that a plurality of paper surfaces do not correspond to the same paper surface (No in S1103), the control unit 104 advances the process to S1105. In step S1104, the control unit 104 integrates a plurality of paper surface images determined to correspond to the same paper surface into one paper image. Here, one paper image obtained by the integration is referred to as an integrated image. Moreover, the process of S1104 is an example of an integration process. In the example shown in FIGS. 13B and 13C, the page image 1012 is integrated into the corrected page image 1303.
In step S <b> 1105, the control unit 104 stores the obtained paper image in the storage unit 105. At this time, the control unit 104 may further store relationship information indicating the parent-child relationship between the corrected paper image (integrated image) and the paper image integrated with the corrected paper image. For example, when the erroneously detected paper surface image 1012 is a region of a photograph inserted into the paper surface, it is possible to easily perform processing such as cutting out and storing the photograph. Here, the process of S1105 is an example of an association process for storing a plurality of object images before integration in association with the integrated image.
As another example, in S1105, the control unit 104 may store the position information instead of the paper image or together with the paper image, as described in S306 in the first embodiment. . In this case, the control unit 104 stores the relationship information indicating the parent-child relationship between the position information also in the relationship information. The other configuration and processing of the image processing apparatus according to the second embodiment are the same as the configuration and processing of the image processing apparatus according to the first embodiment.

  As described above, when the image processing apparatus according to the second embodiment erroneously detects one sheet of paper as a plurality of paper surfaces, the paper image is corrected and erroneously detected according to a user operation. The corresponding paper image can be deleted at the same time.

(Third embodiment)
FIG. 14 is a diagram for explaining an image processing apparatus according to the third embodiment. As shown in FIG. 14A, when two paper surfaces 1400 and 1401 are placed, as shown in FIG. 14B, one paper region is erroneously detected and a paper image 1410 is projected. To do. In this case, the image processing apparatus according to the third embodiment divides one sheet image corresponding to one detected sheet area into a plurality of sheet areas in response to a user operation.
FIG. 15 is a flowchart illustrating a paper surface detection process performed by the image processing apparatus according to the third embodiment. In FIG. 15, the same processes as those described with reference to FIG. After correcting the paper image in S305, in S1501, the control unit 104 determines whether one paper image corresponds to a plurality of papers. Specifically, the control unit 104 specifies the difference area of the paper image before and after the correction, and further specifies the area of the difference area.

Then, when the area of the difference area is equal to or larger than the reduction amount threshold, the control unit 104 determines that the erroneous detection is caused by determining that two sheets of paper are one sheet of paper. It is determined that it corresponds to the paper space. For example, in the paper image correction process (S305), it is assumed that the paper image 1410 shown in FIG. 14B is corrected (reduced) to the paper image 1600 shown in FIG. In this case, the control unit 104 specifies the difference area 1601 of the paper image before and after correction, and further specifies the area of the difference area 1601. Here, the area of the difference area is an example of a reduction amount, and the process of S1501 is an example of a reduction amount specifying process for specifying a reduction amount when a paper image as an object image is reduced.
Returning to FIG. 15, the control unit 104 detects the paper area again in the difference area. Thereby, a paper image 1610 corresponding to the newly detected paper region shown in FIG. 16B is obtained. Other configurations and processes of the image processing apparatus according to the third embodiment are the same as the configurations and processes of the image processing apparatus according to the other embodiments.

  As described above, the image processing apparatus according to the third embodiment detects a plurality of paper surface areas with high accuracy by detecting the paper surface again in the difference area when the paper image is reduced. The boundary of the paper area can be detected correctly.

Next, a modified example of the image processing apparatus according to the third embodiment will be described. When there are two or more sheets to be photographed as in the usage scene assumed in the present embodiment, a sheet that succeeds in detection and a sheet that fails can be mixed. In this case, it is necessary to select and save a paper image corresponding to the paper surface that has been successfully detected, and to correct the paper image for the paper image corresponding to the paper region that has failed to be detected.
However, when a large number of sheets are placed on the projection table 201, it is difficult to perform a user operation for correcting the sheet area or a user operation for selecting a sheet image from a plurality of sheet images. Therefore, the image processing apparatus according to this modification uses a mechanism for selecting a paper image according to an operation for removing the paper by the user. As a result, it is possible to easily select and save a paper image corresponding to the paper that has been successfully detected, and to secure an area on the projection table 201 for correcting the paper image corresponding to the paper that has failed to be detected.

FIG. 17 is a diagram illustrating a flow of selecting a specific paper image from a plurality of paper images by the above method. In this modified example, only the paper on which detection has failed is re-photographed. As shown in FIG. 17A, it is assumed that four sheets of paper 1701 to 1704 are placed on the projection table 201. FIG. 17B is a diagram showing paper surface images 1711 to 1714 projected on the paper surfaces 1701 to 1704 in FIG. As described above, correct paper surface images 1712 and 1714 are obtained for the paper surfaces 1702 and 1704, whereas incorrect paper surface images 1711 and 1713 corresponding to the frames in the paper surface are obtained for the paper surfaces 1701 and 1703. Is obtained.
In this case, the user removes correctly detected paper surfaces 1702 and 1704 from the projection table 201 as shown in FIG. For example, the control unit 104 captures a moving image in the paper area, and determines that the paper has been removed when the total pixel value of the time difference image is equal to or greater than a certain value. Then, the control unit 104 selects the paper image corresponding to the removed paper surface. The control unit 104 further displays markers 1720a and 1720b indicating the selected state. Note that the control unit 104 only needs to display information indicating that the paper surface has been removed, and the specific information for that purpose is not limited to the marker in FIG. As another example, the control unit 104 may display a frame having a color and thickness different from those of other paper images, a captured image obtained based on a paper region, and the like.

Then, after the selected page image is stored in the storage unit 105, the user corrects the positions and areas where the detection success probability of the pages 1701 and 1703 that have failed to be detected is high, as shown in FIG. Place it in a position where it is easy to perform Thereafter, in response to a user operation, the control unit 104 captures the paper planes 1701 and 1703 again and performs a paper plane image correction process.
As a result, when a plurality of sheets of paper are erroneously detected as one sheet, correction of the area and addition of the area can be performed simultaneously by a single process. In addition, when there are simultaneously a paper surface that has been successfully detected and a paper surface that has failed, it is possible to easily select a successful paper surface area.

<Other embodiments>
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads and executes the program.

  As described above, according to each embodiment described above, it is possible to correct the detection area of the object in accordance with a natural touch operation by the user.

  The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to such specific embodiments, and various modifications can be made within the scope of the gist of the present invention described in the claims.・ Change is possible.

101 Projector, 102 Shooting Unit, 103 UI Unit, 104 Control Unit, 105 Storage Unit, 106 Communication Unit

Claims (9)

Photographing means for photographing a photographing region including an object;
Detecting means for detecting an object region from a photographed image photographed by the photographing means;
Projection means for projecting an object image indicating the object area on the object based on the position of the object area in the captured image;
A dividing unit that divides a non-object area other than the object area in the shooting area into a plurality of partial areas based on a position of an outer peripheral line of the object image;
Corresponding means for associating the plurality of partial regions with partial outer peripheries constituting the outer peripheral line;
Target specifying means for specifying, as a correction target, the partial outer periphery associated with the partial area including the input position of the operator on the non-object area;
An image processing apparatus comprising: correction means for correcting the object image by correcting the position of the correction target . A movement amount specifying means for specifying a movement amount of the input position according to the movement of the operation element;
The image processing apparatus according to claim 1 , wherein the correction unit corrects a position of the correction target based on the movement amount. A moving direction specifying means for specifying a moving direction of the input position according to the movement of the operation element;
The image processing apparatus according to claim 1 , wherein the correction unit corrects the position of the correction target in a direction corresponding to the moving direction. When a plurality of object areas are detected and an object image corresponding to the detected one object area is corrected, an overlap between the corrected object image and an object image corresponding to another object area is detected. Overlap detection means;
Determination means for determining whether the plurality of object areas are object areas corresponding to one object based on a detection result by the overlap detection means;
The integration unit according to any one of claims 1 to 3, further comprising an integration unit that integrates the plurality of object images to obtain one integrated image when the determination unit determines that the object area corresponds to one object. The image processing apparatus according to item 1.   The determination unit determines that the one object region and the other object region are object regions corresponding to one object when the overlap detection unit detects an overlap of areas equal to or larger than an area threshold. The image processing apparatus according to claim 4. Wherein when the integrated image is obtained by the integration means, the image processing apparatus according to claim 4 or 5, further comprising an association means for storing a plurality of object images before integration in association with the integrated image. A reduction amount specifying means for specifying a reduction amount when the object image is reduced by the correction means;
The image according to any one of claims 1 to 6, wherein the detection unit detects the object region again in the difference region of the object image before and after correction when the reduction amount is equal to or greater than a reduction amount threshold value. Processing equipment. An image processing method executed by an image processing apparatus,
A shooting step for shooting a shooting area including the object;
A detection step of detecting an object region from a photographed image photographed in the photographing step;
A projecting step of projecting an object image indicating the object area on the object based on the position of the object area in the captured image;
A division step of dividing a non-object region other than the object region in the photographing region into a plurality of partial regions based on a position of an outer peripheral line of the object image;
An associating step of associating the plurality of partial regions with partial peripheries constituting the perimeter line;
A target specifying step for specifying, as a correction target, the partial outer periphery associated with the partial area including the input position of the operator on the non-object area;
The object image is corrected based on a correction means for correcting the object image by correcting the position of the correction target, and an input position of an operator on a non-object area other than the object area in the photographing area. And an image processing method. Detecting means for detecting an object region from a photographed image obtained by photographing a photographing region including an object;
Projection control means for projecting an object image indicating the object area on the object based on the position of the object area in the captured image;
A dividing unit that divides a non-object area other than the object area in the shooting area into a plurality of partial areas based on a position of an outer peripheral line of the object image;
Corresponding means for associating the plurality of partial regions with partial outer peripheries constituting the outer peripheral line;
Target specifying means for specifying, as a correction target, the partial outer periphery associated with the partial area including the input position of the operator on the non-object area;
A program for functioning as a correction means for correcting the object image by correcting the position of the correction target .

Images