JP2005049934A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily detect a position of a specific pattern in an input image. <P>SOLUTION: A tracking pattern 6 is input in an input area 4. Areas for tracking the tracking pattern are set as a tracking area 2a on the left side of a screen and a tracking area 2b on the right side, and the tracking pattern is not tracked in an area except the areas. In the tracking area 2a, a pattern 5a of a left hand is tracked, and operation information about an operation target 3a corresponding to a position of the pattern 5a is output. In the tracking area 2b, a pattern 5b of a right hand is tracked, and operation information about an operation target 3b corresponding to a position of the pattern 5b is output. The operation targets 3a, 3b are displayed on the display screen 1 as virtual operation elements by halftone. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
[Technical field to which the invention belongs]
The present invention relates to an image processing apparatus that targets only a specific area of an input image when tracing a pattern.
[0002]
[Prior art]
Conventionally, theremin is known as a musical instrument to be played without touching the instrument body. Theremin is equipped with an antenna for changing the pitch and an antenna for changing the volume on the main body. By moving the hands around the two antennas, the pitch is specified and the volume is adjusted. While playing.
[0003]
[Problems to be solved by the invention]
As a means of obtaining performance information of an electronic musical instrument that is played without touching the instrument body, a virtual performance operator is displayed on the screen displaying the performer's image, and the movement of the performer's hand is displayed from the input image. It is conceivable to extract performance information corresponding to the movement of the performer's hand by detecting and determining where the detected hand position is in the virtual performance operator.
With such a means, a virtual performance operator is displayed on an electronic musical instrument that is played without touching the instrument body, so that it is possible to perform the performance as if it were operated. become able to. However, detection of position information of a specific pattern in an input image requires a high-speed processing device, and if the operation target is a moving image, it is necessary to perform computation per frame in real time. However, it was too expensive.
[0004]
Therefore, an object of the present invention is to provide an image processing apparatus that can easily detect the position of a specific pattern in an input image.
[0005]
[Means for Solving the Problems]
Since the image processing apparatus of the present invention that can achieve the above object tracks a predetermined pattern only in an area including at least an operation target, it is not necessary to perform the entire analysis of the entire screen. The calculation amount can be greatly reduced. Thereby, the position detection of a predetermined pattern can be performed easily. In this case, for example, an operation target that is an operation element is displayed on the screen in a halftone, so that the predetermined pattern can be easily moved to a desired position of the operation target without being hidden by the operation target. become able to.
In addition, since the predetermined pattern to be tracked is input to the input area, even if the input image fluctuates momentarily due to lighting effects etc., the predetermined pattern input to the input area also changes similarly. Thus, it becomes possible to cope with fluctuations in the illuminance and hue of the input image.
[0006]
DETAILED DESCRIPTION OF THE INVENTION
The image processing executed by the image processing apparatus according to the embodiment of the present invention will be described with reference to FIG. However, FIG. 1 shows an example of a display screen on which a video signal output from the image processing apparatus according to the present invention is reproduced.
In FIG. 1, a tracking area 2a is set on the left side of the display screen 1 on which a moving image 5 as an input image is displayed, and the operation target 3a is mixed with the input image and halftone in the tracking area 2a. It is displayed in transparency. A tracking area 2b is set on the right side of the display screen 1, and the operation target 3b is mixed with the input image and a halftone in the tracking area 2b to be displayed in a translucent manner. The operation target 3a is an operation element for designating a pitch (TONE), and the operation object 3b is an operation element for designating a volume (VOLUME). Further, a small input area 4 is set at the upper center of the display screen 1, and the right hand of the moving image 5 is sampled by inputting and sampling the right hand portion of the moving image 5 in the input area 4 as shown by the broken line. Is set as the tracking pattern 6. This tracking pattern 6 is displayed in the input area 4.
[0007]
In the image processing apparatus, the movement and locus of the pattern matching the tracking pattern 6 is detected by analyzing only the images in the tracking areas 2a and 2b. In this case, the left hand pattern 5a of the moving image 5 matching the tracking pattern 6 is tracked in the tracking area 2a, and the coordinate position of the left hand pattern 5a on the display screen 1 for each frame is calculated. Then, pitch information of the coordinate position of the operation element in the operation target 3a corresponding to the calculated coordinate position is obtained and output. Accordingly, by moving the left hand to a desired pitch position on the pitch operator (operation target 3a), the operation information of the pitch is output. In the tracking area 2b, the right hand pattern 5b of the moving image 5 matching the tracking pattern 6 is tracked, and the coordinate position of the right hand pattern 5b on the display screen 1 for each frame is calculated. Then, volume information of the coordinate position of the operation element in the operation target 3b corresponding to the calculated coordinate position is obtained and output. Thus, by moving the right hand to a desired volume position on the volume controller (operation target 3b), the volume operation information is output. The output pitch information and volume information are supplied to the sound source as performance information, so that a musical sound based on the pitch information and volume information supplied from the sound source is generated and emitted.
[0008]
As described above, the moving image 5 displayed on the display screen 1 is a performer, and the performer can designate the pitch by moving the left hand in accordance with the virtual operator of the pitch on the display screen 1. In addition, the sound volume can be specified by moving the right hand in accordance with a virtual sound volume control on the display screen 1. The operation element (operation object 3a) for designating the pitch can be a keyboard, for example, and the hand can be easily moved to any key, so that the pitch of the musical tone can be designated without error. become. Further, the operator (operation target 3b) for designating the sound volume can be a control panel, and the sound volume can be easily designated as a desired sound volume. In this case, since the operation elements of the operation objects 3a and 3b are mixed with the moving image 5 that is the input image and displayed in halftone, the operation elements such as a keyboard or a control panel of an instrument that is not actually located there. Thus, it is possible to give the player an effect as if he / she is operating, and even a musical instrument which is played without touching the instrument body can be easily played.
[0009]
By the way, if the tracking pattern is a fixed pattern, it may be difficult to analyze a pattern that matches the tracking pattern when an external factor such as illuminance or dimming state of the place changes. Particularly in places where music performances are performed, it is easily expected that the state of the input image will vary considerably from moment to moment due to lighting effects and the like. Therefore, the tracking pattern 6 to be tracked is sampled in the scene or frame unit in the input area 4 shown in FIG. 1, or the tracking pattern 6 to be tracked is always sampled in the input area 4 and used as a matching pattern. To. In this way, it becomes possible to cope with variations in the illuminance, hue, and the like of the input image at the installation location to some extent. In the tracking areas 2a and 2b, the tracking pattern 6 set in the input area 4 is tracked and the position information is analyzed. In this case, it is only necessary to analyze the images in the tracking areas 2a and 2b and to track the tracking pattern 6, so that the amount of calculation is remarkably reduced compared to the analysis of the entire screen of the display screen 1. And the calculation cost can be reduced.
[0010]
Next, another example of the display screen displayed when the image processing apparatus according to the embodiment of the present invention executes image processing is shown in FIG.
In the display screen 1 shown in FIG. 2, about 1/3 of the upper part of the screen is set as the tracking area 2. In the tracking area 2, the operation target 3 is half-mixed with the input image and displayed in a semi-transparent manner. The operation target 3 can be a keyboard or the like for designating a pitch or an operator for designating a volume. Further, a small input area 4 is set at the lower right of the display screen 1, and the moving image 5 is sampled by placing the right hand portion of the moving image 5 in the input area 4 and sampling the input area 4 as shown in the figure. Is set as the tracking pattern 6.
[0011]
In the image processing apparatus, the movement and trajectory of the pattern that matches the tracking pattern 6 is detected by analyzing the image in the tracking area 2. In this case, the left hand pattern 5a of the moving image 5 is tracked in the tracking area 2 in the tracking area 2, and the coordinate position on the display screen 1 of the left hand pattern 5a for each frame is calculated. In this case, the tracking coordinate 7 is surrounded by a rectangle, and for example, the calculated coordinate is the center of gravity of the tracking coordinate 7. Then, performance information such as pitch information or volume information at the coordinate position in the operation target 3 corresponding to the calculated coordinate position is obtained and output. The output pitch information or volume information performance information is supplied to the sound source, so that the pitch and volume of the musical sound generated by the sound source is controlled by the performance information. In this case, in the tracking area 2, the pattern input in the input area 4 is tracked and the position information is analyzed. In this way, since only the analysis of the image in the tracking area 2 needs to be performed and the set tracking pattern 6 needs to be tracked, the calculation amount can be reduced compared with the analysis of the entire screen of the display screen 1. It can be remarkably reduced, and the calculation cost can be reduced. In the example of the display screen 1 shown in FIG. 2, the tracking pattern 6 to be tracked is placed in the input area 4 and is always sampled and used as a matching pattern. Thereby, it becomes possible to cope to some extent with fluctuations in illuminance, hue, and the like of the input image at the installation location.
[0012]
A specific example of the display screen 1 shown in FIG. 2 is shown in FIG.
In the display screen 1 shown in FIG. 3, about 1/3 of the upper part of the screen is a tracking area 2, and the synthesizer front panel and keyboard are halftone mixed and displayed in a semitransparent area in the tracking area 2. Yes. The keyboard in this synthesizer is the operation target 3. In the portion excluding the tracking area 2, the input image is displayed as it is. This semi-transparent synthesizer becomes a virtual instrument played by the performer shown in the moving image 5, and the synthesizer displayed in the tracking area 2 is always at the same position on the screen regardless of the movement of the moving image 5. Is displayed. Further, three input areas 4 are provided, and a tracking pattern to be tracked can be input to each input area 4. In the case shown in the drawing, the upper half of the index finger and middle finger of the right hand is placed in the right input area 4 and sampled to form a tracking pattern 6a. In addition, the tip portion of a stick with a sphere at the tip is placed in the center input area and sampled to obtain a tracking pattern 6b. Furthermore, the tip of the writing instrument is placed in the left input area and sampled to form a tracking pattern 6c. Thus, images of the tracking patterns 6a, 6b, and 6c are displayed in the plurality of input areas 4.
[0013]
Any of the tracking patterns 6a, 6b, and 6c displayed in the plurality of input areas 4 can be selected. Here, it is assumed that the tracking pattern 6a is selected. Then, the pattern tracked in the tracking area 2 is a pattern obtained by replacing the tracking pattern 6a of the right hand with the left hand, and the pattern 5a in the moving image 5 is tracked as a pattern matching the tracking pattern 6a. Accordingly, the movement and locus of the pattern 5a that matches the tracking pattern 6 in the image processing apparatus are detected by analyzing the image in the tracking area 2. That is, the coordinate position of the pattern 5a on the display screen 1 tracked in the tracking area 2 is calculated for each frame. In this case, the tracking coordinate 7 is shown surrounded by a rectangle, and for example, the calculated coordinate is the barycentric position of the pattern 5 a in the tracking coordinate 7. Then, pitch information at the coordinate position of the keyboard that is the operation target 3 corresponding to the calculated coordinate position is obtained and output. That is, the pitch of the key of the keyboard pointed by the middle finger and the index finger of the left hand is output as performance information. The output performance information is supplied to the sound source, so that the pitch of the musical sound generated by the sound source is controlled by the performance information.
[0014]
When the tracking pattern 6b is selected in the input area 4, the tip of the stick with a sphere at the tip becomes the tracking pattern in the tracking area 2, and the pitch of the key of the keyboard pointed to by this stick is played in the same manner as described above. It will be output as information. When the tracking pattern 6c is selected in the input area 4, the tip of the writing instrument becomes the tracking pattern in the tracking area 2, and the pitch of the keyboard key pointed to by the writing instrument is output as performance information in the same manner as described above. It becomes like this.
In the example shown in FIG. 3, the tracking area 2 tracks a pattern that matches the selected tracking pattern among the tracking patterns input in the selected input area 4, and analyzes the position information. In this way, it is only necessary to analyze the image in the tracking area 2 and track the pattern that matches the selected tracking pattern. The calculation amount can be remarkably reduced, and the calculation cost can be reduced.
[0015]
By the way, if the tracking pattern is fixed, it may be difficult to analyze a pattern matching the tracking pattern when an external factor such as the illuminance at the place or the dimming state changes as described above. Conceivable. Therefore, the tracking pattern to be tracked is sampled in the selected input area 4 for each scene or frame, or the tracking pattern to be tracked is sampled in the selected input area 4 and is always used as a matching pattern. To do. Thereby, it becomes possible to cope to some extent with fluctuations in illuminance, hue, and the like of the input image at the installation location.
[0016]
Next, FIG. 4 is a block diagram showing the hardware configuration of the image processing apparatus according to the embodiment of the present invention.
In the image processing apparatus 10 shown in FIG. 4, a CPU (Central Processing Unit) 11 executes an image processing program stored in a memory system (MEMORY SYSTEM) 12, so that a pattern between a tracking pattern and an input image in the tracking area. This is a processing device that outputs operation information corresponding to the movement of the tracking pattern detected by performing image processing such as matching processing. The memory system 12 stores tracking pattern image data input to the input area 4, RAM (Random Access Memory) in which the work area of the CPU 11 is set, and programs such as an image processing program and various data. It is a storage device composed of a ROM (Read Only Memory). The operation information interface (operation information I / F) 13 is an interface for outputting operation information which is performance information obtained in the image processing apparatus 10 to a sound source or the like. This operation information is, for example, a MIDI signal or a signal of a similar format.
[0017]
A plurality of video inputs V1, V2, and V3 can be input to the video input interface 14, and a plurality of video inputs V1, V2, and V3 are switched by a switcher, and an image processing unit ( DISP SYSTEM) 16. At this time, the input analog video signal is converted into a digital video signal by an AD converter (ADC), temporarily stored in the input buffer memory, read out at a predetermined timing, and supplied to the image processing unit 16. The The input digital video signal is temporarily stored in the input buffer memory, read at a predetermined timing, and supplied to the image processing unit 16. The image processing unit 16 is composed of a DISP (Digital Image Signal Processor), and performs a process for halftone mixing a virtual operation element image displayed in the tracking area 2 under the control of the CPU 11 into an input image, The processed image signal is stored in the image memory 15 and image processing such as pattern matching processing is performed. An image signal is read from the image memory 15 at a display timing, converted to an analog video signal by a DA converter (DAC) 17 and supplied to the display as a video signal. As a result, as shown in FIG. 3, the display panel 1 displays the front panel of the synthesizer mixed with the halftone in the input image, the keyboard, and the like. The CPU 11 to the DAC 17 are connected by a bus 18.
[0018]
Next, FIG. 5 shows a flowchart of system processing executed by the image processing apparatus 10 according to the present invention.
When the image processing apparatus 10 is turned on, system processing is started. In system processing executed in real time, processing is performed for each frame, and one frame of data is captured in step S10. Next, in step S11, the tracking pattern set in the input area 4 is detected by performing pattern matching with the image in the tracking area 2, and processing for calculating the coordinate position of the detected pattern is performed. . In step S12, it is determined whether or not a tracking pattern is detected from the tracking area 2 in this process. Here, if it is determined that a tracking pattern has been detected from within the tracking area 2, the process proceeds to step S13, and the area of the operation target 3 in which the center of gravity of the pattern that matches the detected tracking pattern is displayed in a translucent manner It is determined whether it is within.
[0019]
Here, if it is determined that the center of gravity of the pattern to be matched with the tracking pattern is within the area of the operation target 3, the process proceeds to step S14 and image conditions such as hue and brightness near the center of gravity of the pattern to be matched with the tracking pattern. Is determined whether or not the image condition of the set tracking pattern is met. If it is determined that the image condition near the center of gravity of the pattern to be matched with the tracking pattern matches the image condition of the set tracking pattern, the process proceeds to step S15, and the center of gravity of the pattern to be matched with the tracking pattern is Operation information on the coordinate position of the operation target 3 corresponding to the position coordinates is output. If it is determined in step S12 that the tracking pattern is not detected from within the tracking area 2, or if it is determined in step S13 that the center of gravity of the pattern that matches the tracking pattern is not in the area of the operation target 3, If it is determined in step S14 that the image condition in the vicinity of the center of gravity of the pattern to be matched with the tracking pattern does not match the image condition of the set tracking pattern, the system proceeds to step S16 and has already been executed. The operation information being output is reset or the output is ended by the processing. For example, when key-on operation information is being output, key-off operation information is output. If operation information for designating the pitch is being output, the output of the operation information is stopped.
[0020]
When the process of step S15 is completed, or when the process of step S16 is completed, the process returns to step S10, the next frame data is captured, and the above-described steps S11 to S16 are performed on the captured frame data. The process is repeated. In this way, system processing for each frame data is always performed in real time.
In the above description, the image processing apparatus detects performance information. However, the present invention is not limited to this, and can be applied to various parameter setting apparatuses and data input apparatuses.
[0021]
【The invention's effect】
As described above, the present invention traces a predetermined pattern only in an area including at least an operation target, so that it is not necessary to analyze all of the entire screen, and the amount of calculation of image processing is greatly reduced. Will be able to. Thereby, the position detection of a predetermined pattern can be performed easily. In this case, for example, an operation target that is an operation element is displayed on the screen in a halftone, so that the predetermined pattern can be easily moved to a desired position of the operation target without being hidden by the operation target. become able to.
In addition, since the predetermined pattern to be tracked is input to the input area, even if the input image fluctuates momentarily due to lighting effects etc., the predetermined pattern input to the input area also changes similarly. Thus, it becomes possible to cope with fluctuations in the illuminance and hue of the input image.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a display screen displayed when an image processing apparatus according to an embodiment of the present invention executes image processing.
FIG. 2 is a diagram illustrating another example of a display screen displayed when the image processing apparatus according to the embodiment of the present invention executes image processing.
3 is a diagram showing a specific example of the display screen shown in FIG.
FIG. 4 is a block diagram illustrating a hardware configuration of the image processing apparatus according to the embodiment of the present invention.
FIG. 5 is a flowchart of system processing executed by the image processing apparatus according to the present invention.
[Explanation of symbols]
1 Display screen, 2 tracking area, 2a, 2b tracking area, 3 operation target, 3a, 3b operation target, 4 input area, 5 moving image, 5a pattern, 5b pattern, 6 tracking pattern, 6a, 6b, 6c tracking pattern, 7 coordinates during tracking, 10 image processing device, 11 CPU, 12 memory system, 13 operation information I / F, 14 video input interface, 15 image memory, 16 image processing unit, 17 DAC, 18 bus

Claims (4)

Display means for displaying an operation target image on a screen on which an input image is displayed;
A designation means capable of designating a predetermined tracking pattern;
Pattern detection means for tracking a predetermined tracking pattern designated by the designation means and detecting a position of the predetermined pattern on the screen at least in an area where the operation target is displayed;
Output means for outputting operation information of the operation target indicated by the position of the predetermined pattern detected by the pattern detection means;
An image processing apparatus comprising: The image processing apparatus according to claim 1, wherein the operation target is mixed with the input image in a halftone and displayed on the screen. The designation means has an input area set on the screen, and is displayed in the input area by setting the input image to display a part of the input image in the input area. 2. The image processing apparatus according to claim 1, wherein a part of the input image is designated as the predetermined pattern. The designation means has a plurality of input areas set on the screen, and inputs an image pattern to each of the plurality of input areas, and selects one of the plurality of input areas. The image processing apparatus according to claim 1, wherein the image pattern in the selected input area is designated as the predetermined pattern.

Images