JP2009237951A - Presentation system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a presentation system simplifying distinction of a prescribed first processing request such as command operation and a prescribed second processing request such as drawing operation and having an operation method along intuition of a presenter. <P>SOLUTION: This presentation system 1 has: a display surface 2; a display control means 22 displaying an already-created image; a pointer 3; a specific background area 91 or 92; and cameras 4, 5 for photographing a three-dimensional coordinate space. In the presentation system 1, three-dimensional coordinate values of the pointer and those of the specific background area are calculated from an image photographed by the camera, and prescribed first processing or prescribed second processing are performed according to mutual positional relation between the three-dimensional coordinate values of the pointer and the specific background area. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention displays an image on a display body such as a screen or a display, and when a presenter gives a presentation, moves the indicator, performs a command operation such as changing the display image, and adds a line to the display image. It relates to a presentation system that performs processing.

A conventional hand pointing device displays an image showing a three-dimensional space on a display, images a person to be recognized from a plurality of different directions, determines whether a hand of the person to be recognized has a specific shape, Some of the display images are enlarged or reduced when the conditions are met. (For example, refer to Patent Document 1).
JP, 11-134089, A Especially Claim 4

  The conventional handpoint device determines the shape of the hand. Depending on the presenter, there are individual differences in hand size and finger length. Therefore, it is necessary to normalize the recognition target and determine the change, and the arithmetic processing of the apparatus becomes congested. Further, the captured image may change depending on the direction of the hand of the person to be recognized in which direction, and it may be difficult to determine.

  Further, in the presentation, the presenter performs various actions such as explanation, selection of a display image, instruction of a specific position of the image, tracking of the reaction of the audience, and in addition, there is a mental tension. It is preferable that the device used in such a situation is as simple and intuitive as possible.

  An object of the present invention is to obtain a presentation system that simplifies the determination of a predetermined first processing request such as a command operation and a predetermined second processing request such as a drawing operation. It is another object of the present invention to obtain a presentation system that is easy to master operation and that has an operation method in line with the presenter's intuition.

  Other problems of the present invention will become apparent from the description of the present invention.

A presentation system according to one aspect of the present invention includes:
Display surface extending in the X-axis and Y-axis directions An already-created image storage unit that stores an already-created image to be displayed on the display surface,
Pre-created image selection means for selecting a pre-created image stored in the pre-created image storage unit;
Display control means for displaying the already created image selected by the already created image selecting means on the display surface;
Two cameras for photographing a three-dimensional coordinate space including the display surface;
A colored indicator that may move in the three-dimensional coordinate space;
Stores the color data of the indicator, and may be located in a camera field of view captured by the two cameras, and the specific background for a specific background region that is a partial region in the camera field of view Indicator color background color storage unit for storing color data of the area;
An indicator background mutual position storage unit for storing a fixed positional relationship between the indicator and the specific background region;
A photographed image storage unit for storing photographed images taken by the two cameras at the same time;
Have
Call the photographed image stored in the photographed image storage unit, detect the indicator image position of the same color as the indicator color stored in the indicator color background color storage unit, and calculate the three-dimensional coordinate value of the indicator Coordinate position detection means for detecting a specific background image of the same color as the background color stored in the indicator color background color storage unit and calculating a three-dimensional coordinate value of the specific background area;
Instruction for determining whether or not the mutual positional relationship between the three-dimensional coordinate value of the indicator and the three-dimensional coordinate value of the specific background region satisfies a certain positional relationship stored in the indicator background mutual position storage unit Body background mutual position determination means,
When the indicator background mutual position determination means satisfies a certain positional relationship, a predetermined first process is performed,
When the pointer background mutual position determination means does not satisfy a certain positional relationship, a predetermined second process is performed.

  The present invention determines the background color of the background around the indicator, and performs a predetermined first process when the background color is a specific color. A predetermined second process is performed when the background color is other than the specific color.

  Examples of the predetermined process include a drawing process on an image displayed on the display surface, a page advance page return of an already created image, a so-called animation start / stop in which a display image changes, and a sound generation stop.

  In the presentation system according to a preferred embodiment of the present invention, the specific background area may be a partial area of the outline area of the display surface on which the created image is displayed.

  In this preferred embodiment, the color of the contour portion of the display surface is used as the background color. For example, the color of the wall can be used as the contour portion. When the presenter requests a predetermined first process, the presenter moves the indicator along the outline (outside) of the already created image display surface. When a predetermined second process is requested, the indicator is moved within the display surface. For this reason, the presentation system further matches the presenter's intuition.

  In the presentation system according to another preferred embodiment of the present invention, the specific background area may be a partial area of the display surface area on which an already created image is displayed.

  This preferred embodiment uses a specific color in the display surface as the background color. For example, a green part is created on the right side of the slide in the slide display surface. Regardless of which slide is displayed, if the indicator is placed on the green portion, a predetermined first process is performed. Therefore, the presentation system further matches the presenter's intuition.

  In the presentation system according to another preferred embodiment of the present invention, the predetermined first process is a process of instructing the change to the already-created image selection unit or the display control unit, and the predetermined second process May be a drawing process for adding a drawing image to the image displayed on the display surface.

  The preferred embodiment performs command processing and drawing processing.

In a presentation system according to another preferred embodiment of the present invention, the predetermined second process is a drawing process, and the presentation system further includes:
An indicator coordinate value storage unit for storing a three-dimensional coordinate value of the indicator;
With reference to the three-dimensional coordinate value stored in the coordinate value storage unit, and having a drawing means for generating a drawing image according to the three-dimensional coordinate value,
The display control means is for displaying the drawing image on the display surface simultaneously with the selected pre-created image,
The coordinate value calculation means calculates the three-dimensional coordinate values x1, y1, and z1 of the indicator at time t1, and the coordinate values are stored in the coordinate value storage unit. , X2, y2, and z2 that are three-dimensional coordinate values of the indicator at time t2 after a predetermined time has elapsed from time t1, and the coordinate values are stored in the coordinate value storage unit,
The drawing image generated by the drawing means has an image type determined according to the value of z1 that is the Z-axis coordinate value at time t1, and corresponds to the values of x1 and y1 that are the X-axis and Y-axis coordinate values at time t1. Then, the drawing start point is determined, and the drawing end point is determined in accordance with the values of x2 and y2 which are the X-axis and Y-axis coordinate values at time t2.

  According to this preferred embodiment, in the drawing process, the drawing image is corrected depending on the depth position of the indicator, and the corrected drawing is added to the display surface. The presentation system can be operated in accordance with the presenter's intuition and habits, such as by drawing more prominently in the part that the presenter wants to emphasize.

  The present invention described above, preferred embodiments of the present invention, and components included in these can be implemented in combination as much as possible.

  According to the present invention, the predetermined first process and the predetermined second process request are determined based on the background color. Therefore, the determination of the processing request is simplified and the system operation is simplified. As a result, improvement in followability, low cost of the system, and the like are achieved. On the other hand, if the indicator is operated at a specific color portion, a predetermined first processing request is obtained, and if the indicator is operated on the image, a predetermined second processing request is obtained, and the presenter's operation matches the intuition. And has the effect of simplifying the operation.

  Hereinafter, the presentation system 1 according to the embodiment of the present invention will be further described with reference to the drawings. The dimensions, materials, shapes, relative positions, etc. of the members and parts described in the embodiments of the present invention are not intended to limit the scope of the present invention only to those unless otherwise specified. It is just an illustrative example.

  FIG. 1 is a configuration diagram of the presentation system 1. The presentation system 1 includes a display surface 2, which is a screen, an indicator 3, a first camera 4, a second camera 5, and a system controller 8.

  The image display in this embodiment is performed by the projector 6 and the screen. However, in the present invention, the display surface 2 is not limited to the screen, and may be a display surface such as a liquid crystal display, for example.

  The display surface 2 is a plane that extends in the X-axis direction and the Y-axis direction. In the present invention, the plane includes a plane defined by geometry and a curved surface according to the characteristics of the projector and the display. In the present invention, the X axis and the Y axis represent a relative relationship with the Z axis that expresses the distance from the display surface.

  The first camera 4 and the second camera 5 are cameras that capture a three-dimensional space including the display surface 2, and may be a moving image camera or a still image camera. However, in order to perform smooth drawing or the like, it is preferable to obtain an image at a speed of, for example, about 60 frames / second or more. Therefore, the first camera 4 and the second camera 5 are preferably moving image cameras. For example, a CCD camera can be used as the first camera 4 and the second camera 5.

  The first camera 4 and the second camera 5 preferably have the entire surface of the display surface 2 as the field of view, and preferably include the contour region of the display surface 2 as shown in the field of view 7 in FIG. The number of cameras is not limited to two and may be three or more.

  The indicator 3 is, for example, a red or green object. The shape of the object is, for example, a spherical body or a cube. In order to facilitate the operation of the presentation system 1 by moving the indicator 3, the indicator 3 is preferably attached to the tip of the bar. Also, the presenter may wear, for example, a red or green glove on one hand and use the glove as the indicator 3.

  The indicator 3 is a means for operating the presentation system 1. The indicator 3 moves within the field of view of the first camera 4 and the second camera 5 or stops. Also. Move out of the field of view or move still. The indicator in the visual field is detected, and the processing of the presentation system 1 proceeds.

  The indicator 3 may be positioned in front of the specific background area (91 or 92). The specific background area (1) 91 is an example in which the area is provided on the display surface. The specific background region can be provided by specifically coloring a part of the slide that is an already created image. In this case, if there are a plurality of slides, it is preferable to provide a specific background region colored in the same color of all the slides in a portion common to all the slides (for example, the upper left corner). The specific background area (2) 92 is an example in which the area is provided outside the display surface. For example, the wall of the presentation place can be a specific background area.

  The system controller 8 is, for example, a computer. The system controller 8 includes an already created image storage unit 11, an indicator color background color LUT 12 (lookup table) that is an indicator color background color storage unit, a three-dimensional coordinate storage unit 13, a lens correction coefficient storage unit 14, and an image-space. Conversion coefficient storage unit 15, movement distance value storage unit 16, drawing image type storage unit 17, drawing correction storage unit 18, depth value storage unit 19, indicator background mutual position storage unit 20, photographed image storage unit 31, lens corrected Image storage unit 32, indicator coordinate value storage unit 33, background coordinate value storage unit 34, already created image selection unit 21, display control unit 22, coordinate value calculation unit 23, drawing unit 24, first operation determination Means 25, command control means 27, target color determination means 27, depth determination means 30, indicator background mutual position determination means 51, a main control unit 28, and a peripheral device control unit 29.

  Each of the above means can be realized by a program incorporated in a computer, a CPU, a RAM, a ROM, and the like. Each said memory | storage part is realizable by allocating the fixed part in the hard memory of a computer, for example. Furthermore, the already created image selection means 21 can use a commercially available presentation system program, for example.

  The main control unit 28 performs overall operation control of the system controller 8. The peripheral device control unit 29 performs operations of the first camera 4 and the second camera 5, image processing, control of the projector 6, and the like. Further, the system controller 8 includes a system display device 41 and an input device 42. The system display device 41 is a liquid crystal display attached to the computer, and the input device 42 is a keyboard and a pointer, for example, a mouse.

The indicator color background color LUT and the indicator or background color discriminating method in the camera photographed image used in this embodiment are as follows:
Toshikazu Wada: Color Target Detection Information Processing Society Transaction “Computer Vision and Image Media” with Nearest Neighbor Discriminator Vol. 44, No. SIG17-014, 2002
Has been reported.

(Initial setting)
The initial setting operation of the presentation system 1 will be described.

  An already created image to be displayed on the display surface 2 is created and stored in the already created image storage unit 11. The already created image is a character, a figure, a photograph, an illustration, a mixture of these, or the like. The already created image may be a blank image. A specific example of the created image is a slide document for presentation.

  The drawing image type is stored in the drawing image type storage unit 17. The drawing image types are, for example, a solid line, a broken line, an alternate long and short dash line, a circle mark, a triangle mark, a square mark, and an arrow. A single drawing image type may be stored.

  The drawing correction storage unit 18 stores the correction items of the drawing image. The correction items include, for example, line thickness, color shading, and hue. Then, a relational expression with the value of z1 is described. For example, when correcting the line thickness, when 0 ≦ z1 <50, the line thickness is 100 pixels, when 50 ≦ z1 <100, the line thickness is 70 pixels, and when 100 ≦ z1 <150 The line thickness is described as 40 pixels. For example, if the hue is to be corrected, it is described as red when 0 ≦ z1 <100, orange when 100 ≦ z1 <200, and yellow when 200 ≦ z1 <300.

  The indicator background mutual position storage unit 20 describes the relationship between the detected coordinate value of the indicator and the detected coordinate value of the specific background region. If the operation of positioning the indicator 3 in the specific background area portion is performed, both Z-axis values are close to each other, and both X-axis values and Y-axis values are also close to each other. A positional relationship such as the proximity relationship may be described in the storage unit.

  For example, if the display surface is a reference plane of Z = 0 and a specific background area exists in the specific background area plane of Z = n that is a plane parallel to the reference plane, the indicator background mutual position storage unit 20 The X-axis value and the Y-axis value of the detected coordinate value of the pointer are within the specific background area, and the Z-axis value of the detected coordinate value of the pointer and the Z-axis value of the detected coordinate value of the specific background area are determined in advance. The relationship that the positional relationship is satisfied if the proximity is within the predetermined value is stored.

  The indicator background mutual position storage unit 20 may store only a fixed value that is a distance between the detected coordinate value of the indicator and the detected coordinate value of the specific background area. In this case, it is not a strict judgment as to whether or not the indicator exists in the specific background area, but if it is set to determine that the positional relationship is satisfied when sufficiently close, there is no problem in operation.

(Environmental setting)
Next, environment setting is performed. The environment setting is a setting in which the presentation system 1 is installed in an actual use place and necessary data is input to the presentation system 1 according to the lighting state, the indicator movement range, and the like.

  The presentation system 1 is installed at the presentation place, and the display surface 2 and the projector 6 are arranged. The 1st camera 4 and the 2nd camera 5 are arrange | positioned, and the camera visual field 7 is adjusted. The camera field of view is fixed during environment setting and during subsequent steady operation.

  Next, the presentation system 1 is taught the recognition target color.

  FIG. 2 is a diagram for explaining the contents of the indicator color background color LUT12, and FIG. 3 is a flowchart of the recognition target color teaching process.

  The indicator color background color LUT12 has a plurality of color channels. The first camera has a color channel Ch1-1 used for indicator color discrimination and Ch1-2 which is a color channel used for background color discrimination for a captured image, and stores a recognition target color and a non-recognition target color for each channel. . The second camera has a color channel Ch2-1 used for indicator color discrimination and Ch2-2 which is a color channel used for background color discrimination for a captured image of the camera, and a recognition target color and a non-recognition target color for each channel. Remember.

  In S11, the indicator 3 is positioned in the camera visual field 7, and the specific background area (1) 91 is displayed on the display surface 2, and the first camera 4 and the second camera 5 capture an image including the indicator 3. . When the specific background area (2) 92 is used, the indicator 3 may be positioned in the camera visual field 7 and the camera visual field 7 may be photographed. In S 12, the captured image of the first camera 4 and the captured image of the second camera 5 are stored in the captured image storage unit 31.

  In S13, the captured image of the first camera 4 is called from the captured image storage unit 31 and displayed on the system display device 41. In S14, the indicator 3 area on the system display device is designated by the input device 42. By this operation, the first indicator color C1-1 is taught. In S15, the first indicator color C1-1 is stored in the indicator color background color LUT12.

  In S16, a non-indicator region is designated for the same image displayed on the system display device 41. By this operation, the first non-recognition target color NC1-1-1 is taught. In S17, the first non-recognition target color NC1-1-1 is stored in the indicator color background color LUT12. S16 and S17 are repeated to teach other non-recognition target colors NC1-1-n.

  Here, when there is no non-recognition target color to be taught in the same image displayed on the system display device 41 in S16, the following photographing operation is repeated. That is, an image including the specific non-recognition target color is displayed on the display surface 2, or an object of the specific non-recognition target color is positioned in the camera visual field 7 and photographed with the first camera, and S12, S13 , S16, S17.

  As described above, the recognition target color C1-1 of the indicator, the non-recognition target colors NC1-1-1, and NC1-1-2 of the indicator are taught about the first camera photographed image, and the instructions It is stored in the body color background color LUT12. In FIG. 2, the color channel Ch1-1 is displayed.

  Next, the process proceeds to S181, and the same process of S14 to S18 is repeated for the background color of the first camera photographed image. By this processing, the background recognition target color C1-2, the background non-recognition target color NC1-2-1, NC1-2-2 --- NC1-2-n are taught for the first camera photographed image, and the indicator It is stored in the color background color LUT12. In FIG. 2, the color channel Ch1-2 is displayed.

  In S19, the same processing as S13 to S18 is repeated for the second camera image. The second indicator color C2-1 specified in S14 and the first indicator color C1-1 described above are substantially the same, but are colors that reflect the color characteristics of the second camera and the first camera, respectively. It is data. The background color C2-2 and the background color C1-2 have the same relationship.

  When the above recognition target color teaching is completed, as shown in FIG. 2, the recognition target color and the non-recognition target color are stored for the color channels Ch2-1 and Ch2-2, respectively.

  In a presentation system including three or more cameras, a pointer color and a non-recognition target color, a background color and a non-recognition target color are taught for a third camera photographed image and the like.

  Next, with reference to FIG. 4, the environment setting for image-space conversion will be described.

  In S21, the image on which the reference point for correction is drawn is displayed on the display surface 2. A similar printed material may be attached to the display surface instead of the image display. The image on which the correction reference point is drawn is, for example, a grid image or a checkered pattern image. Then, the display is photographed by the first camera 4 and the second camera 5.

  In S 22, the captured image of the first camera 4 and the captured image of the second camera 5 are stored in the captured image storage unit 31. In S23, the captured image of the first camera 4 is called from the captured image storage unit 31 and displayed on the system display device 41. An image coordinate value of a reference point for correction on the display image is detected.

  In S24, the reference point for correction in the three-dimensional coordinates (world coordinates) is associated with the image coordinate value detected in S23. In S25, the three-dimensional coordinate value and the image coordinate value are stored in a temporary memory. The operations of S23 to S25 are repeated for other correction reference points.

  In S26, the three-dimensional coordinate value group and the image coordinate value group corresponding to each one-to-one are called from the temporary memory, and the lens correction coefficient of the first camera photographed image and the image-space conversion coefficient of the first camera photographed image are obtained from both groups. calculate. The lens correction coefficient is a coefficient value, and the image-space conversion coefficient is a matrix value.

  In S <b> 27, the lens correction coefficient of the first camera image is stored in the lens correction coefficient storage unit 14. In S 28, the image-space conversion coefficient of the first camera image is stored in the image-space conversion coefficient storage unit 15.

  In S29, the same operation as in S23 to S28 is performed on the image captured by the second camera. With this operation, the lens correction coefficient of the second camera photographed image is stored in the lens correction coefficient storage unit 14, and the image-space conversion coefficient of the second camera photographed image is stored in the image-space conversion coefficient storage unit 15.

(Steady operation-1)
Next, the steady operation process will be described with reference to the flowchart of the steady operation shown in FIG. 5 and the three-dimensional coordinate determination processing flowchart of the indicator 3 and the specific background region shown in FIGS. 6 and 7.

  Steady operation is started in S40. During steady operation, the indicator 3 moves or stops according to the operation of the presenter.

  In S <b> 41, the already created image selection unit 21 selects one image from the already created image storage unit 11.

  In S <b> 42, the display control unit 21 displays the image on the display surface 2. In S440, the first camera 4 and the second camera 5 simultaneously photograph each field of view, calculate the three-dimensional coordinate value of the indicator 3 from the position of the indicator 3 in the photographed image, and also in the photographed image. The three-dimensional coordinate value of the specific background area is calculated from the position of the specific background area. Detailed processing of S440 will be separately described below.

  With this calculation, the three-dimensional coordinate values x1, y1, and z1 of the indicator 3 at the time t1, the three-dimensional coordinate values of the specific background area, and the coordinate values x2, y2, and z2 at the time t2 are calculated. These three-dimensional coordinate values of the indicator 3 are stored in the indicator coordinate value storage unit 33. Further, the three-dimensional coordinate value of the specific background area is stored in the background coordinate value storage unit 34.

  In S441, the indicator background mutual position determination unit 51 calls the three-dimensional coordinate value of the indicator 3 at time t1 from the indicator coordinate value storage unit 33. Also, the three-dimensional coordinate value of the specific background area at time t1 is called from the background coordinate value storage unit 34. Then, the indicator background mutual position determination means 51 compares the relationship between the two three-dimensional coordinate values with the positional relationship stored in the indicator background mutual position storage unit 20.

  The determination performed by the indicator background mutual position determination means 51 is recorded in the indicator coordinate value storage unit 33 when, for example, all the detected background coordinate value groups are stored in the background coordinate value storage unit 34 as background regions. Is there a combination of coordinates that calculates the positional relationship between the determined detection indicator coordinate point and the detected background coordinate value for all combinations of points and satisfies the positional relationship stored in the indicator background mutual position storage unit 20? Determine whether or not. This method may be used when the determination is made only with a fixed value of the distance between the detected coordinate value of the indicator and the detected coordinate value of the specific background area.

  Alternatively, when the display surface is the reference surface and the specific background region plane where the background region exists is known to exist parallel to the reference surface, the maximum value of the X-axis value and the Y-axis value of the detected background coordinate value group If the minimum value is obtained by calculation, it is determined whether or not the determined detection indicator coordinate point exists between the X-axis value, the maximum value of the Y-axis value, and the minimum value of the detected background coordinate value group. The positional relationship stored in the pointer background mutual position storage unit 20 by determining whether or not the distance between the Z-axis value of the pointer coordinate point and the Z-axis value of the specific background region plane satisfies the positional relationship. Can be determined.

  In addition, when the background area boundary point that is the outermost part of the detected background coordinate value group is determined by the coordinate value calculation means, and the plane surrounded by the background area boundary point is determined as the specific background area, the background area boundary point The specific background area plane where the specific background area exists is obtained by calculation using a space vector method, etc., the distance between the specific background area plane and the fixed detection indicator coordinate point, the specific background on the specific background area plane Whether or not the positional relationship stored in the indicator background mutual position storage unit 20 is satisfied is determined based on whether or not the detected indicator coordinate point determined inside the region connected with the normal of the region boundary exists. judge. This technique is effective when the specific background area plane is not horizontal to the reference plane.

If the relationship between the two three-dimensional coordinate values at time t1 satisfies the stored positional relationship, the process proceeds to S442. If the relationship between the two three-dimensional coordinate values at time t1 does not satisfy the stored positional relationship, the process proceeds to S500.

  S442 is a start process of a predetermined first process, and in the present embodiment, the predetermined first process is a command control process. That is, in S443, the command control unit 26 performs a predetermined process on the already created image currently displayed or the already created image selection unit. The predetermined process may be, for example, a process in which the next created image is displayed on the display surface 2 by the display control means 22 instead of the already created image being displayed. Further, for example, a process of enlarging or reducing an image being displayed may be performed.

  S500 is a start process of a predetermined second process, and in the present embodiment, the predetermined second process is a drawing process.

  In S51, the drawing unit 24 selects one drawing image type from the drawing image type storage unit 17. For example, the drawing image type may be programmed in advance so that the same drawing image type is always selected during one steady operation, and the drawing image type is determined according to the values of x1, y1, and z1. May be programmed to be selected.

  In S52, the drawing unit 24 refers to the indicator coordinate value storage unit 33 and corrects the drawing image according to the value of z1. When the drawing image type is a line, for example, when the value of z1 is relatively close to the display surface, image correction is performed so that the line is relatively thick, and the value of z1 is relative to the display surface. When the image is far from the image, the image is corrected so that the line is relatively thin.

  In S <b> 53, the drawing unit 24 refers to the values of x <b> 1 and y <b> 1 in the indicator coordinate value storage unit 33 and determines the starting point of the drawn image. In S54, the drawing unit 24 refers to the x2 and y2 values in the indicator coordinate value storage unit 33 to determine the end point of the drawn image.

  In S55, the drawing means calls the already created image that is selected in S41 and that is currently displayed on the display surface 2 from the already created image storage means 11. A process of superimposing the drawing image created and corrected in S51 to S54 on the already created image is performed. The drawing process may be such that the drawn image is displayed in the foreground and the previously created image that is overlapped is not visible, or the drawn image may be drawn with a transparency of 50%, for example. Then, the image is sent to the display control means 22.

  In S <b> 56, the display control unit 22 displays the superimposed image on the display surface 2.

  When S56 ends, the data of x2, y2, and z2 in the indicator coordinate value storage unit 33 are defined as new x1, y1, and z1.

(Steady operation-2, three-dimensional coordinate determination processing)
The three-dimensional coordinate of the indicator 3 and the three-dimensional coordinate determination process of the specific background area in S440 will be described.

  In S61, the first camera 4 and the second camera 5 photograph the fixed visual field simultaneously. In S 62, the captured image of the first camera 4 and the captured image of the second camera 5 are stored in the captured image storage unit 31.

  In S63, the captured image of the first camera 4 is called from the captured image storage unit 31. Using the lens correction coefficient of the first camera photographed image stored in the lens correction coefficient storage unit 14 in S64, lens correction is performed on the first camera photographed image. In step S65, the first camera lens corrected image is stored in the lens corrected image storage unit 32.

  In S66, the same process as in S61 to S65 is performed on the image captured by the second camera. The lens correction coefficient used for this processing is the lens correction coefficient of the second camera photographed image. Accordingly, the second camera lens corrected image is also stored in the lens corrected image storage unit 32.

  In S67, the first point in the three-dimensional coordinate space stored in the three-dimensional coordinate storage unit 13 is extracted. The coordinate value of the extracted point is called “extracted three-dimensional coordinate value”. The three-dimensional coordinate space is a space that includes the display surface 2 (expanding in the X-axis and Y-axis directions) and extends forward (in the direction in which the presenter and audience are located (Z-axis)) and backward from the display surface. At the same time, the three-dimensional coordinate space is a space photographed with the camera visual field 7. The three-dimensional coordinate storage unit 13 stores coordinate points on a space obtained by dividing the three-dimensional coordinate space into a minute cube or a minute rectangular parallelepiped. Although the Z-axis positive / negative of the three-dimensional coordinate value is defined in the right-handed coordinate system, the positive / negative of the Z-axis may be corrected as necessary.

  In S <b> 68, the coordinate value calculation unit 23 uses the image-space conversion coefficient of the first camera photographed image stored in the image-space conversion coefficient storage unit 15 to extract the extracted three-dimensional coordinate value of the first camera photographed image. Convert to coordinate values. The converted image coordinate value is referred to as “extracted image coordinate value”.

  In S69, the target color determination unit calls the first camera lens corrected image, and extracts the extracted image while referring to Ch1-1 and Ch1-2 which are color channels for the first camera photographed image of the indicator color background color LUT12. It is determined whether or not the pixel of the coordinate value corresponds to the recognition target color. If it falls under the recognition target color, the process proceeds to S70. If the color does not correspond to the recognition target color, the process returns to S67, the second point in the three-dimensional coordinate space stored in the three-dimensional coordinate storage unit 13 is extracted, and the same processes of S68 and S69 are performed.

  In S70, the same processing as S68 and S69 is performed on the second camera lens corrected image. At this time, the image-space conversion coefficient used for conversion of the extracted image coordinate value is an image-space conversion coefficient of the second camera photographed image. Further, the color data in the indicator color background color LUT12 referred to by the target color determination means are Ch2-1 and Ch2-2 which are color channels for the second camera photographed image.

  In S <b> 71, the target color determination unit 27 determines whether or not a recognition target color is detected in the first camera captured image and the second camera captured image. That is, when the pixel of the extracted image coordinate value in the second camera photographed image is determined as the recognition target color, the extracted three-dimensional coordinate value is regarded as the detected coordinate point of the indicator 3 or the specific background region. In the case of the indicator color, the extracted three-dimensional coordinate value is stored as a detection indicator coordinate point in the indicator coordinate value storage unit 32. In the case of the background color, the extracted three-dimensional coordinate value is stored in the background coordinate value storage unit 33 as the detected background coordinate point. When the recognition target color is not detected in the second camera photographed image, the process returns to S67, the second point in the three-dimensional coordinate space stored in the three-dimensional coordinate storage unit 13 is extracted, and the same processing as in S68 and S69 is performed. I do.

  In S72, the processing from S67 to S71 is repeated to perform recognition target color detection processing at all points in the three-dimensional coordinate space.

  In S73, if the target color is detected in the above processing, the process proceeds to S74. If no target color is detected in the above processing, the process proceeds to S76.

  In S74, the detected indicator coordinate value group stored in the indicator coordinate value storage unit 33 is determined as one point by the coordinate value calculation means. For example, the center of gravity of the detected coordinate value group may be obtained, or the coordinates of the point closest to the display surface may be selected from the detected coordinate value group.

  Next, a background area is determined from the detected background coordinate value group stored in the background coordinate value storage unit 34. All the detected background coordinate value groups stored in the background coordinate value storage unit 34 may be stored as a background area, and a background area boundary point corresponding to the outermost part of the detected background coordinate value group is determined by the coordinate value calculation means. A plane surrounded and formed by the boundary points of the background area may be determined as the specific background area. The background area boundary point includes the background area vertex and is composed of three or more points.

  In S75, the determined detection indicator coordinate point is stored in the indicator coordinate value storage unit 33 as x1, y1, and z1 at time t1. Further, the determined background area is stored in the background coordinate value storage unit 34.

  In S76, the process returns to S61, and pointer coordinate detection is performed on the captured image in the next frame (time t2). Thus, the coordinate points x2, y2, z2 of the indicator at time t2 are stored in the indicator coordinate value storage unit 33, and the background area at time t2 is stored in the background coordinate value storage unit 34.

  If the presentation system has three or more cameras, the processing similar to S68 and S69 for the second camera lens corrected image in S70 is followed by the same processing as S68 and S69 for the third camera lens corrected image. Etc.

  In a presentation system using a huge display surface 2, the environment setting for image-space conversion is performed by the above-described operation, and the three-dimensional coordinates to be extracted are calculated in the indicator three-dimensional coordinate value during steady operation. The value range may be limited to the range in which the indicator moves. In this way, it is possible to obtain an effect such as an increase in calculation speed.

(First modified embodiment)
The presentation system 1 (first modified example) to which the depth position determination of the indicator as a preferred embodiment is added will be described. The added depth position determination is a process of determining whether or not to perform a predetermined second process based on the depth position of the indicator in the three-dimensional space.

  According to this preferred embodiment, when the indicator is located within a certain range of depth, the predetermined second process is performed. For example, when the presenter approaches the display surface and moves the indicator, drawing is performed. On the other hand, when the presenter leaves the display surface and moves the indicator near the podium, for example, drawing is not performed.

  In general, a presenter has a habit of moving in a certain depth direction, for example, by approaching a display surface and tracing the display surface with an indicator when explaining an already created image such as a slide. Therefore, this preferred embodiment is a presentation system that further conforms to the inventor's habits when the predetermined second process is a drawing process.

  However, the process performed after the depth position determination in the present invention is not limited to the drawing process.

  FIG. 8 is a flowchart for explaining the first modified embodiment. In the first modified embodiment, the processes of S81 to S82 are inserted between S500 (predetermined second process start) and S51 (drawing start) in the steady operation described above.

  In the depth value storage unit 19, the Z-axis threshold value of the indicator is stored in advance. Alternatively, both end values of the Z axis are stored. In the case of the threshold value, it is described that the process proceeds to the drawing process when z1 is larger (or smaller) than the threshold value. In the case of both-end values, it is described that the process proceeds to the drawing process when z1 is between the end-values (or when it is outside). The values and conditions stored in the depth value storage unit 19 are collectively referred to as “threshold conditions”.

  In S <b> 81, the depth determination unit 30 compares the value stored in the depth value storage unit 19 with the value of z <b> 1 stored in the indicator coordinate value storage unit 33. If z1 satisfies the threshold condition in S82, the process proceeds to S51 to perform drawing processing. If the z1 value is outside the threshold condition, the process returns to S440.

(Second modified embodiment)
The presentation system 1 (second modified example) to which the moving speed judgment of the indicator as another preferred embodiment is added will be described. The added moving speed determination is a process of determining whether or not to perform a predetermined second process based on the moving speed of the indicator in the three-dimensional space.

  According to this preferred embodiment, when the indicator moves at a specific speed, the predetermined second process is performed. In general, the presenter has a habit of slowly moving the indicator when requesting an explanation and drawing of an already created image such as a slide. On the other hand, there is a habit of quickly moving the indicator when a slide feed or the like is requested.

  Therefore, in this preferred embodiment, when the predetermined second process is a drawing process, the presentation system further conforms to the inventor's habits. However, the process performed after the depth position determination in the present invention is not limited to the drawing process.

  FIG. 9 is a flowchart for explaining the second modified embodiment. In the second modified embodiment, the processes of S91 to S92 are inserted between S500 (predetermined second process start) and S51 (drawing start) in the steady operation described above.

  A movement distance value of the indicator is stored in advance in the movement distance value storage unit 16.

  In S91, the first operation determination means 25 calculates the distance (calculated movement distance) between the position of the pointer at time t1 and the position of the pointer at time t2 stored in the pointer coordinate value storage unit 33. In S92, the first operation determination unit 25 compares the calculated movement distance with the movement distance value stored in the movement distance value storage unit 16.

  When the calculated movement distance is smaller than the stored distance value, the process proceeds to S51 to start the drawing process. If the calculated movement distance is smaller than the stored distance value, the process returns to S440.

The value stored in the movement distance storage unit 16 and the determination performed by the first operation determination unit 25 are, for example,
(1) A single threshold value is stored and compared with the calculated travel distance.
(2) Memorize both end values of a single range, and compare and judge whether the calculated moving distance falls within the range,
(3) Store both end values of 2 in 2 ranges, and compare and determine which range the calculated moving distance falls into,
It may be.

1 is a configuration diagram of a presentation system 1. FIG. It is a content explanatory drawing of indicator color background color LUT12. It is a flowchart of a recognition target color teaching process. It is a flowchart of the environment setting concerning image-space conversion. It is a flowchart of steady operation. It is the 1st three-dimensional coordinate determination process flowchart 1 of the indicator 3 and a specific background area | region. It is the flowchart 3 of the three-dimensional coordinate determination process of the indicator 3 and a specific background area | region. It is a flowchart of the depth position determination addition part of the indicator. It is a flowchart of the moving speed judgment addition part of the indicator 3.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Presentation system 2 Display surface 3 Indicator 4 1st camera 5 2nd camera 7 Camera visual field 8 System controller 12 Indicator color background color LUT which is a indicator color background color memory | storage part
91 Specific background area (1)
92 Specific background area (2)

Claims (5)

Display surface extending in the X-axis and Y-axis directions An already-created image storage unit that stores an already-created image to be displayed on the display surface,
Pre-created image selection means for selecting a pre-created image stored in the pre-created image storage unit;
Display control means for displaying the already created image selected by the already created image selecting means on the display surface;
Two cameras for photographing a three-dimensional coordinate space including the display surface;
A colored indicator that may move in the three-dimensional coordinate space;
Stores the color data of the indicator, and may be located in a camera field of view captured by the two cameras, and the specific background for a specific background region that is a partial region in the camera field of view Indicator color background color storage unit for storing color data of the area;
An indicator background mutual position storage unit for storing a fixed positional relationship between the indicator and the specific background region;
A photographed image storage unit for storing photographed images taken by the two cameras at the same time;
Have
Call the photographed image stored in the photographed image storage unit, detect the indicator image position of the same color as the indicator color stored in the indicator color background color storage unit, and calculate the three-dimensional coordinate value of the indicator Coordinate position detection means for detecting a specific background image of the same color as the background color stored in the indicator color background color storage unit and calculating a three-dimensional coordinate value of the specific background area;
Instruction for determining whether or not the mutual positional relationship between the three-dimensional coordinate value of the indicator and the three-dimensional coordinate value of the specific background region satisfies a certain positional relationship stored in the indicator background mutual position storage unit Body background mutual position determination means,
When the indicator background mutual position determination means satisfies a certain positional relationship, a predetermined first process is performed,
A presentation system that performs a predetermined second process when the pointer background mutual position determination means does not satisfy a certain positional relationship.   The presentation system according to claim 1, wherein the specific background area is a partial area of a contour area of the display surface on which an already created image is displayed.   The presentation system according to claim 1, wherein the specific background area is a partial area of the display surface area on which an already created image is displayed.   The predetermined first process is a process of instructing the pre-created image selection unit or the display control unit to change, and the predetermined second process adds a drawing image to the image displayed on the display surface. The presentation system according to claim 1, wherein the presentation system performs drawing processing. The predetermined second process is a drawing process, and the presentation system further includes:
An indicator coordinate value storage unit for storing a three-dimensional coordinate value of the indicator;
With reference to the three-dimensional coordinate value stored in the coordinate value storage unit, and having a drawing means for generating a drawing image according to the three-dimensional coordinate value,
The display control means is for displaying the drawing image on the display surface simultaneously with the selected pre-created image,
The coordinate value calculation means calculates the three-dimensional coordinate values x1, y1, and z1 of the indicator at time t1, and the coordinate values are stored in the coordinate value storage unit. , X2, y2, and z2 that are three-dimensional coordinate values of the indicator at time t2 after a predetermined time has elapsed from time t1, and the coordinate values are stored in the coordinate value storage unit,
The drawing image generated by the drawing means has an image type determined according to the value of z1 that is the Z-axis coordinate value at time t1, and corresponds to the values of x1 and y1 that are the X-axis and Y-axis coordinate values at time t1. The presentation system according to claim 1, wherein the drawing start point is determined, and the drawing end point is determined corresponding to the values of x2 and y2 which are the X-axis and Y-axis coordinate values at time t2.

Images