JPH11205654A - Image display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To correct the position of a rotated image by detecting the rotation deviating amount of the shape of an image between images in series of images of an image pickup device or input device. SOLUTION: This display device displays the erect image of a rotated image by correcting a coordinate to a reference coordinate corresponding to the rotation deviating amount of a first rotation deviating amount detecting means 7c for detecting the rotation deviating amount of a shape provided by tracing due to a first tracing means 7b based on the reference shape of a first reference designating means 7a for setting features such as the coordinate of the shape on a picture to extract the shape from the specified color information of picked-up image, the position of the center of gravity and direction as references for tracing.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a photographed image of an electronic still camera or a video camera or the like.
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image display device that detects a change in the direction of a specific subject image on a screen and corrects the rotation direction of a display image to form an erect image, in connection with signal processing of a series of recorded images and the like.

[0002]

2. Description of the Related Art As an example of a conventional image display device capable of changing the display direction of an image, Japanese Patent Laid-Open Publication No. Hei 5-3536 is an example.
In Japanese Patent Application Laid-Open No. 4 (1999) -2000, setting of an image display direction by an operator's key operation is disclosed.

In Japanese Patent Application Laid-Open No. Hei 7-168529, a plurality of optical sensors are arranged around an image display device to detect the position of an operator with respect to the display device.
The image display direction is controlled so that the display of the display device as viewed from the operator is always an erect image.

[0004]

However, the above-described prior art image display apparatus detects a direction in which the display should be corrected by a key operation by an operator, movement to an optical sensor, or the like, and detects the detected direction. , The direction of the image display is set according to the following formula. However, high-speed response may be difficult due to the configuration. Further, as disclosed in Japanese Patent Application Laid-Open No. H8-319069, a display device of a portable information communication device is provided with a gravity sensor for detecting the direction of gravity, and the output of the gravity sensor controls the image display direction. A device has been proposed by the applicant.

[0005] As an example of this gravity sensor, a conductive liquid is sealed in a container having a smaller volume than the container, and the conductive liquid is moved in the direction of gravity to be stable. There is a configuration in which the rotation angle is detected by detecting conduction / non-conduction due to contact / non-contact between a plurality of opposing electrodes provided in advance around the inside of the container and a conductive liquid located at a stable position thereof. In order to correct the rotation direction about the optical axis of the camera, in a display device using a gravity sensor, the angular speed of image rotation is about 1 rotation / 1 rotation.
In the case of the second, that is, the frequency, it is possible to follow up if the frequency is about 1 Hz or less, but there is a drawback that the response becomes difficult at a higher speed than that due to the structure of the sensor.

[0006] In recent years, as a video camera, an electronic still camera, or the like, a camera having a configuration in which a camera unit and a display unit are separated and connected by a dedicated cable has been put on the market.

[0007] In these, it is possible to place the display unit at hand and move the camera unit to a free position so that the photographer can take a picture from a direction that is easy to see.

However, in such a configuration, since the camera section has good mobility and can rotate freely with respect to the optical axis of the lens, the captured image is easily rotated. For this reason, the camera operator shoots the image while rotating the camera as little as possible while watching the display of the image, or performs the operation while correcting the direction of the original erect image even if the camera is rotated. But,
For example, when the camera operator and the person observing the display are different, the camera operator cannot see the displayed image, so it is difficult to control the rotation angle of the image, and the size of the rotation angle and its speed are unspecified. Easy to fluctuate.

Therefore, when the correction is performed by the above-described gravity sensor, the rotation speed of the camera is easily increased depending on the operation of the camera, and a high-speed area that cannot be corrected by the gravity sensor may be generated. . For this reason, the following operation becomes poor, and the image is rotated, making it difficult to see. Furthermore, if you follow a rotating image,
It is also conceivable to fall into a situation such as feeling sick.

The present invention solves such a conventional drawback for a viewer of a display screen, and changes the direction of the shape of an erect image of a specific color designated as a reference on the screen to the direction in which the rotation is fixed. As a reference, while detecting the shape specified as the reference, detecting the amount of rotational deviation in the direction of the tracked shape, correcting the screen coordinates for rotation, and displaying a continuously upright image on the display screen. It is intended to provide a configuration to be realized.

[0011]

In order to achieve the above object, the present invention sets the position of the image pickup device so that an erect image can be obtained on a display screen when photographing. Next, the observer of the display screen specifies the color of the erect image at the same time as or before the observation, determines the luminance corresponding to the color of the specified erect image, and responds on the imaging screen. It has a shape extracting means for extracting a binary shape of an erect image specified by an observer in real time by extracting pixels of color and luminance. Furthermore, reference designating means for designating and setting features such as the coordinates of the shape on the screen, the position of the center of gravity, the direction, and the like as the reference for tracking, and extracting the set reference shape. Tracking means to track in real time, the set reference shape, a specific point from the center of gravity of the same shape obtained by tracking it, or, by tracking the reference shape by the direction of a specific straight line portion A rotational displacement detecting means for calculating a rotational displacement difference between the obtained shape and the same shape to detect a rotational displacement of the image, and an image in pixel units in the direction of the reference shape based on the detected rotational displacement. The invention comprises a correction unit for performing the rotation correction of the above, and a unit for recording an output image from the correction unit or displaying the corrected image and constantly displaying an erect image to an observer.

Further, in order to cope with panning and the like more widely, when the same shape obtained by tracking the reference shape moves to a specific area on the screen or when a predetermined amount of movement is detected, the camera is moved. Extract another new reference shape in the direction, the coordinates of the shape on the screen obtained by the extraction,
A reference re-designating means for setting the characteristics such as the position and direction of the center of gravity as a reference and repeating the operation of setting the new reference, and tracking while extracting the reference shape, a new reference shape, etc. A second tracking means, a second rotational displacement detecting means for detecting a rotational displacement from the same shape obtained by tracking, and a reference shape at the time of setting; A second correction unit for performing a rotation correction of the image based on the correction.
The invention is constituted by means for recording the image or displaying an image to constantly display an erect image to an observer.

Further, the present invention can be applied to not only image input from a camera held by a photographer. Therefore, even if the target image is a series of output images recorded on the recording image device from an electronic still camera, an ultrasonic scanner, an X-ray camera, or the like, an input / output means for inputting these and outputting thereafter is provided. And an image via this is processed in the same manner as described above, and an upright image is always displayed to the observer.

In addition to the rotation of the image due to the rotation of the imaging device, it is assumed that the position of the display device and the position of the observer change, and this is corrected. Therefore, when the display surface of the display device is individually rotated by, for example, 90 degrees, a rotation amount instructing means for manually or automatically detecting and instructing the rotation amount is provided,
Next, the rotation output amount of the image processed in the same manner as described above and the rotation amount of the rotation amount instructing means are input, and a calculation output to be corrected by the correcting means so that an erect image is always displayed to the observer is obtained. A configuration having an operation means for outputting is provided.

[0015]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a first embodiment of an image display device according to the present invention will be described.
The embodiment will be described below. FIG. 1 is a block diagram showing the first embodiment.

In FIG. 1, reference numeral 1 denotes an optical system for guiding light passing through a lens to an image pickup device, and 2 denotes an image pickup device for converting a light signal into an electric signal. Reference numeral 3 denotes a CDS for reducing noise of an image sensor, 4 denotes an A / D for converting an analog signal into a digital signal, 5 denotes a camera signal processing unit that processes digital image data by dividing it into colors and luminances, and 6 denotes a RAM.
(Random Access Memory) and a frame memory FM for recording a frame image, a RISC-CPU for controlling the operation and transmission / reception of data of each unit, and performing a program operation at high speed, and a reference numeral 7a for an initially extracted shape. Is designated as reference coordinates, and 7b is a first tracking means for determining a shape having the same area on the screen as the shape set as the reference coordinates, and tracking the reference shape. A first direction detecting means for detecting the direction of a specific point from the center of gravity of the coordinate value of the reference shape of the first reference specifying means 7a and the direction of the specific point from the center of gravity of the coordinate value of the output shape of the first tracking means 7b; A first rotational deviation amount detecting means for calculating a rotational deviation amount of the tracking shape with respect to the reference shape from the detection direction, and a first rotational deviation amount detecting means for calculating the first rotational deviation amount from the first rotational deviation amount detecting means. First performing a coordinate correction of the output shape
Coordinate correction means, 8 is a shape extraction means for extracting a designated color and the luminance of the designated color, 10 is a memory control for controlling processing such as writing or reading a processed image in an image memory, and 11 is coordinate correction. An image memory for recording the images obtained, a key 14 integrally representing various operation instruction keys, a remote control receiver 15, and a color liquid crystal display means 16 for displaying a video signal as a color image.

Next, the operation will be described. The light image from the optical system unit 1 is received by the CCD 2 and converted into an electric signal. The signal outputs a signal reduced in noise by the CDS 3 and is input to the A / D 4 to convert an analog video signal into a digital video signal S 0. The converted signal S0 is input to the camera signal processing unit 5 and corrected by the control signal S1 from the RISC-CPU 7, and outputs a digital video signal S4 of the data and a digital video signal S5 delayed by 1H from the data S4. I do. RISC-CPU7
Inputs an instruction signal of a key 14 or a remote control receiver 15 for receiving a signal of a remote control (not shown), and gives an operation instruction to each unit. Also, the output signal S of the camera signal processing unit 5
Reference numeral 3 denotes a reference clock signal for setting operation timings from the image pickup devices CCD2 to A / D4. Further, the output signal S2 is a signal for controlling the optical system unit 1.

Next, the shape of the erect image in the photographing screen is extracted,
An operation of designating a reference shape from the extracted shapes will be described. The image pickup device 2 is fixed at a position where an erect image can be obtained (a position where a captured image from the screen of the color liquid crystal display means 16 can be seen upright), and one output digital video signal S4 of the camera signal processing unit 5 is It is input to the shape extraction means 8. According to the color designation from the RISC-CPU 7, the level of the luminance signal of the color is determined and binarized to extract the shape. The extracted shape is a binarized data group of 1 and 0 corresponding to the position of the pixel having the designated color on the photographing screen. RISC the binary data
Input to the CPU 7;

As an example of a method of specifying a color, for example, a key corresponding to an individual color may be selected from the keys 14 and a color of a shape to be specified may be input, or the details will be described later. Utilizing the configuration of color extraction of the shape extracting means 8, 12 to 126 colors of only the central part of the screen are simultaneously compared in parallel, and the color of the determination result is automatically set to the designated color. Is also good.

The operation of the RISC-CPU 7 will be described below with reference to the schematic diagram of FIG. 3A and the flowcharts of FIGS. The binarized data is used as first reference designating means 7
When the input is input to the a, the first reference designating means 7a records all positions of the extracted shape of the designated color as coordinates on the screen and performs an operation of designating one of the extracted shapes as a reference.
First, at step 801 in FIG. 7, since it is necessary to first perform the setting operation of the reference shape, the frame flag P indicating the instruction is set to 1. Next, step 80
Then, the process goes to step 2 to set the coordinates of the extracted shape on the screen in the vertical direction.

In FIG. 3a, a vertical address from top to bottom on the screen is obtained.

In order to set the address, a vertical synchronizing signal and a horizontal synchronizing signal which are continuously output are used. Normally, when a video signal captured by an interleave scanning method is obtained, a vertical synchronizing signal and a horizontal synchronizing signal of the circuit also correspond to interleave scanning. Therefore, by utilizing this, when the trailing edge of the vertical synchronization signal (the rising edge in the case of a negative drive signal, the same applies hereinafter) and the leading edge (falling edge) of the horizontal synchronization signal coincide in the odd field. A description will be given below.

The V counter is started from the trailing edge of the vertical synchronizing signal, starts counting at the trailing edge (rising edge) of the horizontal synchronizing signal, and then counts at both edges of the horizontal synchronizing signal. Counting is stopped at the leading edge of the vertical synchronizing signal. This aims at completing all extraction processing in one frame, obtaining odd counter values in odd fields and even counter values in even fields, and interleaving scanning and arrangement when one frame is completed. This is to make each horizontal line correspond to each counter value.

By this operation, after the trailing edge of the horizontal synchronizing signal is counted (odd number), the binary shape in the horizontal direction is output. If the count value at the time when the extracted shape is detected is stored, The vertical position of the extracted shape is determined.

In FIG. 3A, the horizontal direction is assumed to be the horizontal direction from left to right in the figure.

Next, in the even field, since the trailing edge of the vertical synchronizing signal does not match the leading edge of the horizontal synchronizing signal, the V counter is started from the trailing edge of the vertical synchronizing signal, and the leading edge of the horizontal synchronizing signal is started. Start counting at the edge,
Thereafter, counting is performed at both edges of the horizontal synchronization signal, and counting is stopped at the leading edge of the next vertical synchronization signal, thereby generating an even-numbered counter value. After the trailing edge of the horizontal synchronizing signal is counted (even number),
Since the binarized shape is obtained, the position of the extracted shape in the vertical direction can be determined by storing the count value at that time.

An address is obtained by oddly counting the coordinates of the odd-numbered frame image on the screen in the vertical direction, and an address by counting the even-numbered coordinates of the even-numbered frame image on the vertical screen. Is obtained, and a process of recording the coordinates in the vertical direction on the screen subjected to the interleave scanning is performed.

In step 804, the H counter is started from the trailing edge of the horizontal synchronizing signal of the video signal having the above-mentioned shape to obtain the horizontal address from left to right of the screen of FIG. Is counted, and the counter values of the rising edge and the falling edge of the binary data are recorded, and are converted into horizontal coordinate values on the screen. The resetting of the H counter is performed at the leading edge of the next horizontal synchronization signal.

In step 805, the coordinate values of the shapes extracted in steps 802 to 804 are put together, and the coordinate values of the binarized shape obtained by combining the value of the extracted H counter and the value of the V counter at that time into a RISC- CPU7
In a RAM 1 (not shown). That is, a value in the V direction of a shaped horizontal line and a value in the H direction at both ends of the shaped horizontal line are recorded as data of the binarized shape area.

As described above, in the video signal of the interleave scanning, one frame is composed of two fields, the odd-numbered field counts the V-counter value of the vertical coordinate in the odd-numbered field, and the even-numbered field counts the even-numbered V counter value. Since the V counter values from both fields are sorted in order, the extracted shape represented by the vertical value in the image of one frame and the horizontal value from the H counter value is rearranged. A data group of coordinate values, that is, a table of coordinate values of the extracted shape is obtained on the RAM 1.

In the next step 806, it is determined whether or not the shape extracted on the screen is a shape having a set area. For this reason, it is determined whether adjacent horizontal lines on the screen are adjacent to each other on the horizontal lines detected as the binarized data based on the color specification between the upper and lower horizontal lines.

The horizontal line currently being processed is referred to as the current line, and the immediately preceding horizontal line is referred to as the previous line. This will be described in further detail below.

First, the extraction area on the horizontal line indicated by the values at both ends of the previously recorded extraction shape on the current line is:
If it is in contact with any of the extraction regions on the horizontal line indicated by the values at both ends of the previously recorded extraction shape on the previous line, it is determined that the region is a united region.

The determination as to whether or not there is contact is made when the values at both ends of the extraction area of the current line are included in either or both of the two ends of the extraction area of the previous line.
Alternatively, it can be easily determined by detecting a case where the values at both ends of the extraction region of the previous line are included between the values at both ends of the extraction region of the current line.

If it is determined that there is an area in contact with the area after the above determination, it is determined that the area is part of the same shape, and the next step 807 attaches the same label to the area of the previous line. If no, the process goes to step 808 without doing anything.

If there is an extraction area in the current line but no extraction area in the previous line, a new label is assigned, and the flow shifts to step 808.

Until the end of one frame is detected, the process returns to the step 802, where the same operation is continued again, and the coordinate recording process for all the extracted shapes is completed. In the next step 809, the pre-processing is performed by the frame counter P =
1 is determined. YES because P = 1 is initially set
Then, the flow shifts to 810, where the coordinate value of the shape near the center of the screen of the designated color portion is labeled and recorded in the RAM 2 (not shown) of the RISC-CPU 7 in order to indicate the reference coordinate. Further, the reference coordinate setting completion flag is set to the reference coordinate F = 1, and since the setting of the reference shape is completed, the frame counter P = 1 is changed to the frame counter P = 2.

Here, the coordinate value for shape detection of the designated color portion is set near the center of the screen, but is not limited to this, and may be another portion, for example, a shape from the right end of the screen. Also, as will be described in detail later, if a specific area is set and special processing is performed there, it is necessary to avoid that place. It is good anywhere.

Next, the processing shifts to the first tracking means 7b shown in step 902 in FIG. 8, but since the above-mentioned processing is a processing for designating a reference shape and the reference coordinate F is set to 1, the processing in step 901 is executed. The determination as to whether or not the reference coordinate F is 1 is YES, and the flow proceeds to step 904 of the next first direction detecting means without performing the tracking processing.

In the process of step 904, the coordinate value of the center of gravity of the reference shape on which the coordinates have been recorded is obtained by calculation, and the coordinate value of the longest point or the shortest point from the center of gravity is calculated and obtained. In order to obtain the coordinate value of the center of gravity, for example, the coordinates in the horizontal and vertical directions of the region where the extracted shape is present are integrated over the entire screen, and the result is divided by the number of pixels of the existing region. Further, an angle from the horizontal line to a specific point such as the shortest point or the longest point is calculated from the horizontal line, and the result is recorded in the RAM 2 as a reference angle for displaying an erect image.

Further, since the processing instruction of the tracking shape is given from the magnetic field, the reference coordinate F is set to 0 to indicate the instruction.

In the above description, the calculation process from the calculation of the center of gravity to the calculation of the angle can be performed in a short time because it is a simple calculation process based on the coordinate values of each pixel. If the extracted shape has a circular shape or a target shape, and the longest point or the shortest point cannot be calculated, the next image is determined, and the reference coordinates are changed to a shape having the longest point or the shortest point. What is necessary is just to set a process.

Since the above processing is processing for designating a shape, processing of a frame image prior to a frame image currently being processed is not performed. Therefore, the first
In step 905 of the rotational deviation detecting means 7c, the rotational deviation becomes zero, and in the next step 1001 of the first coordinate correcting means 7d in FIG. 9, the processing passes without processing, and the extracted image is recorded in the image memory 11. Move on to processing.

Here, a detailed circuit configuration of the shape extracting means 8 will be described below with reference to FIG. Reference numeral 21 denotes a normalizing unit for normalizing the color saturation, and reference numerals 22 and 23 denote the RISC-CPU 7.
The extraction unit A and the extraction unit B determine a luminance level having a color component according to the color instruction and perform binarization. Numerals 24 and 25 are a logical sum unit A and a logical sum unit B for performing a logical sum operation of the digital signal. 26 and 27 are composed of a filter section A and a filter section B which output a digital signal of one pixel by majority decision of digital binarized data of 3 * 3 pixels.

Next, the operation will be described. Digital video signal S
The luminance and chrominance signals of No. 4 are input to the normalization unit 21 to normalize the color saturation. When the standardized signals are simultaneously input to the extraction unit A22 and the extraction unit B23, the RISC-C
By determining the level of the luminance having the specified color from the PU 7 and binarizing it, a binarized signal based on the specified color is output.

For example, when four colors are designated at the same time, the binarized signals of the designated four colors are output from the extraction units A22 and B23. The output of these four binarized signals is the logical sum A
24, and are input to the logical sum unit B25 and logically ORed to obtain binary signals synthesized based on the designated four colors. The output of the filter section A2
6, and input to the filter section B27 to remove noise and input to the RISC-CPU 7. in this way,
With a simple hardware circuit configuration, binarization processing of a color-designated shape can be performed in real time.

On the other hand, the digital video signal S5 output from the camera signal processing unit 5 (here, it is assumed that the signal has been delayed by 1H for the purpose of delaying the signal processing) is stored in the memory controller
0, the image information is input to the frame memory FM6, and the image information for one frame is temporarily stored.

Here, as described above, the extracted image input to the RISC-CPU 7 at the time of setting the reference coordinates is the first frame image, so that it is stored as it is, and is stored in the frame memory FM6 by the memory control 10. Is stored in the image memory 11 at a memory address corresponding to the coordinates of the extracted image on the screen.

The access path of the frame memory FM6 and the image memory 11, that is, the path for transmitting / receiving data to / from the memory, the instruction for writing / reading of the memory, and the call of the memory address, etc. Description will be made assuming that the path overlaps with the path of the video signal S7.

Next, when a new frame image is obtained as a digital video signal S5 from the camera signal processing section 5, the image data of the previous frame recorded in the frame memory FM6 is replaced with the newly obtained frame image. Replace with data.

The digital video signal S4, which is a new frame image from the camera signal processing unit 5, is adjusted by 1H earlier to adjust the delay generated when performing signal processing such as extraction processing. Is input to the shape extracting means 8, a binarized shape is extracted based on the previously specified color based on the digital video signal S4, and this data is input to the RISC-CPU 7.
To the first reference designating means 7a.

The coordinates on the screen of the shape set as the reference change with the movement of the camera, but recording processing of all the coordinate values of the extracted binarized shape is performed for each frame in the same manner as described above. And the result is RAM of RISC-CPU7.
1 is recorded.

Next, when returning to the processing of FIG. 8, the determination of the reference coordinate F = 1 in step 901 is as follows.
As described above, since F = 0 is set in step 904, the result is NO, and the first tracking means 7 shown in step 902 is set.
Move to b. Therefore, from the RAM 1, the RAM
A shape having an area equivalent to the shape area set as the reference of the two labels A is searched, and the shape obtained by the search is determined to be the same as the reference shape, and this is recorded in the RAM 3.

At this time, if there are a plurality of shapes having the same area, the closest one based on the maximum value in the horizontal and vertical directions may be selected as a simple determination means.
Further, it is needless to say that the shape at the position closest to the position where the previous extracted shape was present may be selected, and it is needless to say that the final shape may be determined by pattern matching, but details are omitted.

Now, in step 903 indicating the operation of the first direction detecting means, the barycenter calculated from the coordinate value of the current frame image and the coordinate value of the shortest or longest distance point from the barycenter are obtained. An angle centered on the center of gravity is calculated and recorded in the RAM 3. Also, the next time, an instruction to perform the processing of the tracking shape is issued, so that the reference coordinate F is set to 0 for the instruction.

Next, in step 905 showing the operation of the first rotational displacement detecting means 7c, since the center of gravity and the shortest point or the longest point of the previous frame image have been obtained from the RAM 3, the reference shape in step 904 is obtained. Based on the center of gravity and the shortest point or the longest point of the label A of the RAM 2
The value of the amount by which the image is translated is obtained from the difference between the coordinate values of the centers of gravity of the respective shapes stored in the AM2 and the RAM3. Furthermore, the shortest point from the center of gravity of RAM2 and RAM3,
Alternatively, the direction of the shortest point or the longest point from the center of gravity of each shape, that is, the value of the angle, is obtained from the coordinate value of the longest point and the coordinate value of the center of gravity. From these, the difference in angle between the frame of the reference shape and the extracted shape of the current frame image, that is, the rotation angle, is obtained, and this is the amount of rotation deviation.

As described above, the amount of rotational displacement of the current frame image having the same shape can be calculated in a short time based on the value calculated by the coordinates on the screen.

Here, when the shape that has been tracked as being the same as the shape set as the reference moves and a part of the shape gradually becomes a shadow and is lost, tracking becomes impossible. . In such a case, the operation and configuration may be such that the shape set as the reference is updated at intervals of 2 to 3 frames, the difference in the shape that has gradually changed is reduced, and even if there is a slight amount of difference. It goes without saying that it is good.

Next, the amount of rotation deviation from the center of gravity is determined by the first coordinate correcting means 7d, which is operated in step 1001 in FIG. 9, and the coordinate recording value of the extracted shape of the current frame image is changed. As a result, the coordinates corresponding to the amount of rotation deviation about the center of gravity of the reference shape are corrected via the memory control 10 and are recorded in the image memory 11 within the vertical blanking period. The rotation of the image may be performed, for example, one pixel at a time, and the rotation may be performed around the center of gravity in the RAM 3.

As described above, the amount of rotation deviation is detected, and the coordinate recording value is changed based on the detected value to perform correction for returning to an erect image.

By the above-described processing, the operation of maintaining the orientation of the image initially set as the reference is performed, and these operations are continued even after the output of the next frame image. Rotation can be suppressed, and an image that always becomes an erect image can be continuously displayed and recorded.

At this time, if the address to be changed and corrected does not exist on the recorded screen, it is needless to say that the correction may be performed by interpolating based on the image data of the current frame image. Omitted.

As described above, in the image in which the reference shape of the set erect image exists, even if the camera freely rotates, the image memory 11 which always obtains the moving image of the erect image can be obtained.
Is recorded and displayed on the color liquid crystal display means 16.

In the above description, the rotational direction of the shape set as the reference is always corrected to prevent the rotation. However, it is obvious that the correction amount is reduced and the correction amount is adjusted to increase or decrease the speed. By preventing rotation for rotation, relaxing the correction for a certain amount of rotation or more, changing the reference angle itself for correction,
It goes without saying that the rotation prevention control can be performed more naturally.

Next, the operation of FIG. 1 will be schematically described with reference to FIG. FIG. 3 shows an image from the camera, FIG.
3d shows the case where the camera is rotated.

FIG. 3A shows an image captured by a camera.
This shows a case where a tree on the image is determined as an erect image. On the screen, two colors, green and brown, are extracted at the same time.
The binarized luminance edge of the large tree 31 and the small tree 32 is extracted as shown by b. In FIG. 3, the coordinates of the edge portion of the extracted image are detected by the first direction detecting means 7c.
The center of gravity G1 is calculated as follows. Further, the longest point M1 is calculated from the center of gravity G1. In the case of this example, the point farthest from the center of gravity is indicated by the ease of explanation when using the figure.
It is set as a reference point for detecting the direction.

Based on the reference point of the extracted shape, an angle K1 between a line segment from the center of gravity G1 to the longest point M1 and a horizontal line passing through the center of gravity G1 is calculated by the first rotational deviation amount detecting means 7c. Next, it is assumed that an image in a state where the camera is moved and rotated is observed on a monitor separated from the camera, as shown in FIG. 3D.

In this figure, the large tree 31 is shown as an erect image in the figure, but this is the first reference designating means 7a.
From the first rotation deviation amount detection means 7c to the first rotation deviation amount detection means 7c, the object is not moved, and the camera is moved, and the camera is moved upright. Is taken.

An image in which the shape of the two trees is obliquely obtained is obtained on the photographed screen. Based on this image, the line segment from the center of gravity G2 to the longest point M2 and the center of gravity G2 are determined. An angle K2 with the horizontal line passing is obtained.

Next, FIG.
The direction from the center of gravity G2 of d to the longest point M2 coincides with the direction from the center of gravity G1 of FIG.
Change the recorded value of the coordinates on the screen and store it in the image memory 11
To record. By changing the coordinate values, the coordinates of the image are corrected as shown in FIG. 3g and recorded in the same direction as the initially set reference erect image (FIG. 3c). An erect image is always displayed and is finally recorded.

As described above, even if the photographed image is rotated, the same shape as the subject set as the reference shape is extracted, and the amount of rotation deviation is detected, and correction is performed based on the extracted result. Like the first frame image with the set shape, subsequent frame images are always
An upright image can be displayed.

Further, since the reference shape can be extracted in real time and the configuration is simple, there are advantages such as a short processing time.

Note that the rectangular image obtained by photographing depends on the shape of the image pickup area of the image pickup device 2, but the rectangular image shown in FIG.
In some cases, there is no image at the corner of g, which is schematically shown in black. Further, the image sensor is NTS
If it is used for C, but if an image sensor for PAL is used and part of it is used as an image output part for NTSC,
Since the number of pixels is about 20% larger than that for NTSC, the black screen can be reduced accordingly. Furthermore, if an image sensor having an increased number of pixels is used, the black portion can be further reduced.

Here, the image rotation processing may be performed as follows. For example, using the coordinates of the center of gravity of the extracted shape as the reference for rotation, calculating the distance from the center of gravity of each picture element, rotating and moving from the center of gravity by the rotated angle, setting it as new coordinates, and using the coordinates of the picture element data Moving. By these operations, the position of the image is rotated around the center of gravity by an amount corresponding to the rotational deviation, and an image that is always erect can be displayed.

Since a series of image information obtained by a VTR, an ultrasonic scanner and an X-ray camera is a video signal similar to the above-mentioned photographed image, input / output means for inputting the image information and thereafter outputting the image information An erect image can be always displayed by processing the output signal (not shown) through the A / D 4 in the same processing, operation, and configuration as described above.

Next, a second embodiment will be described with reference to FIG. In the same figure, those having the same functions as those in FIG. 1 to FIG. 3 and FIG. 7 to FIG.

In FIG. 4, reference numeral 7e includes a reference shape of the first reference designating means. A new reference shape of the same color or a different color is extracted from another location than the same shape, and a new reference coordinate is obtained. And a reference re-designating means for repeating such a new reference coordinate setting operation. 7f is a second tracking means for determining the same area as the reference shape area such as the reference re-designating means 7e or the first reference designating means 7a and performing shape tracking, and 7g is a reference of the reference re-designating means 7e. Second direction detecting means for detecting the direction of a specific point from the center of gravity of the coordinate value and the direction of the specific point from the center of gravity of the coordinate value of the output shape of the second tracking means 7f is incorporated. The calculated second rotational deviation amount detecting means, 7h is the second rotational deviation amount detecting means.
This is a second coordinate correction means for correcting the coordinates of the output shape of the second tracking means 7f based on the rotation shift amount of the rotation shift amount detection means 7g.

Next, the operation will be described. The reference shape described in the first embodiment is obtained by operating the camera and panning,
Alternatively, when the camera moves in the tilt direction, the camera may move in the direction opposite to the pan or tilt direction, and may eventually disappear from the camera shooting screen. The present embodiment is devised as a countermeasure for this.

First, FIG. 10 shows a program flow of the reference re-designating means 7e. In the figure, in step 809, since it is not the first process of setting the reference shape, the determination of P = 1 indicating the reference shape detection instruction is NO, and the second
The process moves to step 812 of the tracking means 7f. The shape having the same area as the label A of the RAM 2 of the data recording of the reference shape is represented by R
The search is performed from AM1, the coordinates are stored in the label M of the RAM1, and the process proceeds to Step 813. This step 81
In step 3, when the reference shape of the label M in the RAM 1 reaches the coordinate area near the end of the screen, the process proceeds to step 814, where the reference shape of the label M is recorded on the label C in the RAM 2, and the center of the screen or the camera movement direction A new reference shape is extracted with the same color in the end area on the side, and is extracted as label B in RAM2.
And the reference coordinate F = 1 of the reference shape setting flag.
Set to.

At this time, although there is no entry in step 814 in the figure, if there is no shape of the set color with respect to the new reference shape, three other colors are set and the same color is set. What is necessary is just to extract a shape. In addition, if there is no set color shape, another color is set, and until the existence can be confirmed.
Needless to say, the color setting may be repeated.

Here, instead of determining whether or not the position of the reference shape of the label M in the RAM 1 in the determination in the step 813 is in the coordinate area near the end side in the screen, the moving distance of the shape straight line is A configuration may be adopted in which a new subject is set as the reference coordinates by detecting that the amount is equal to or more than the specified amount.

If the determination in step 803 is NO,
The process proceeds to step 815, which indicates that the tracking shape is at a position near the center of the screen or the like.
Since it is recorded in M3 and is not the reference coordinate, the reference coordinate F =
The value is set to 0, and the flow shifts to the second direction detecting means in the next step 901 in FIG.

In the determination of the reference coordinate F = 1 in step 901, since the processing is not performed from step 814, the determination is YES.
The process proceeds to step 906 to determine whether or not the label B of the RAM 2 has the new reference shape.

In step 906, the flow advances to step 907 in the direction of YES because the label is the label B in the RAM2.
From the center of gravity of the labels B and C of the RAM 2, the longest point or the shortest point from the center of gravity, the longest point around the center of gravity, or
The angle to the shortest point is calculated, and the process proceeds to step 908 of the next second rotational deviation amount detecting means 7g, where the RAM of the reference shape is stored.
The rotation angle difference R0 due to the camera rotation between the label A of No. 2 and the label C of the tracking shape RAM 2 is detected.

Since the label C of the tracking shape RAM 2 and the label B of the new reference shape RAM 2 are within the same screen, the rotation angle difference R 0 is calculated from the rotation angle R 01 of the label B of the new reference shape RAM 2 (R 01 −R0 = RS)
The reference angle of the label B of the new reference shape RAM2, which is the angle that matches the direction of the initially set reference shape, is set to RS. If the determination in step 901 is NO, the process proceeds to steps 903 and 905, and the rotational displacement amount of the tracking shape is detected.

Then, the flow shifts to step 1002 of the second coordinate correcting means 7h in FIG. 12, and it is determined whether or not the label B is a new reference shape RAM2. If YES, the flow shifts to step 1003 to shift to the new reference shape RAM2. The coordinates of the image are changed in the direction opposite to the moving direction of the rotation angle difference R0 (the angle moved from the angle of the initial reference shape) about the center of gravity of the label B (label B of the RAM 2), and the image is turned into an erect image. to correct. The data of the label B of the RAM 2 is replaced with the label A of the RAM 2, the data storage location of the reference shape is set to one, and the reference shape is changed to a new reference shape.

If NO in step 1002, the flow shifts to step 1001 to calculate the angle difference R0 between the rotational displacement detected by the second rotational displacement detecting means 7g and the new reference shape of the reference re-designating means 7e. Then, the coordinate record value of the rotation of the same shape as the new reference shape is changed. This allows
Each frame image recorded in the field memory FM6 is controlled by the output of the second coordinate correcting means 7h by the memory control 10 and is recorded in the image memory 11, whereby each reference shape or the same shape can be erected.

The next new reference shape, like the new reference shape, may disappear from the screen by moving the camera in the pan or tilt direction, so that it operates in the same manner as described above. Is set, and the rotational angle difference between the set shape and the new reference shape is calculated and detected by the same processing as described above, and is further set to a new reference angle.

By repeating such an operation, new reference shapes can be set one after another in the camera movement direction. Even if the image continuously changes with the movement of the camera, the reference of the initially set erect image can be set. Based on the shape, a new reference shape can be set, and an erect image in the same direction as the first erect image can always be set. Therefore, even if the camera moves and rotates, the photographer or observer can always see the erect image. Is obtained, so that an easy-to-view image is obtained.

As described above, the reference re-designating means 7e, the second tracking means 7f, the second rotational displacement amount detecting means 7g, and the second coordinate correcting means 7h provide a new method when the subject is out of the area of interest. Although the embodiment in which the reference coordinates are set and the tracking and the rotation are corrected have been described, it is needless to say that this embodiment is not the only method for coping with the out of the area. As an example, if the function of the reference re-designating means 7e is used to detect that the shape of the area of interest is out of shape, an appropriate subject in the area is selected as described above, and The label A is attached to the extracted shape of the selected subject and recorded in the RAM 2 in the same manner as the operation of setting the reference coordinates by the first reference specifying means 7a.

As described above, if the selected extracted shape is again used as the reference coordinate, the operation exactly the same as the operation described in the first embodiment is continued, and the rotation of the image is corrected. Operation becomes possible. As described above, if the object selected again is referred to as a unit for setting the reference coordinates, or a reference coordinate resetting unit, this unit is simply added to the embodiment of FIG. The same operation as in the embodiment can be performed.

Next, the operation of the figure will be described in more detail with reference to FIG. FIG. 5A is an extracted image based on the large tree 31. The center of gravity G1, the longest point T1, and the angle R1 are set in the second image.
It is detected by the direction detecting means. Next, when the camera moves rightward, an image is obtained as shown in FIG. 5B. The shape of the large tree 31 in the image is determined by the second tracking means 7f, and the shape is determined by the second direction detecting means to detect a new center of gravity G2, the longest point T2, and the angle R2. 7g
Then, the rotational deviation R1-R2 from FIG. 5A is calculated. Based on this calculated value, the image is rotated by the second coordinate correcting means 7h so that the angle becomes R1 around the center of gravity G2 of the large tree 31 in the broken line frame as shown in FIG. An erect image is obtained by rotation correction.

In FIG. 5B, since the coordinates of the large tree 31 are near the end of the screen, a small tree 33 of the same color in the opposite direction is extracted and set as a new reference shape. The center of gravity G3, the longest point T3, and the angle R3 are calculated. Further, the value and the rotational deviation R1-R2 between FIG.
Difference R3- (R1-R2) between the angle and the angle R3 of the small tree 33
To obtain an angle corresponding to the angle R1 of the first erect image. This becomes the reference angle of the new reference shape.

Next, when the camera moves to FIG.
3 is the center of gravity G4, the longest point T4, and the angle R4, and the angle difference R4-R3- between this value and the reference angle R3- (R1-R2) of the new reference shape set by the small tree 33 in FIG. 5B.
By (R1-R2), the amount of rotation deviation from the initial reference shape can be detected, and the coordinate recording value of the output shape in FIG. Finally, FIG. 5E shows an erect image recorded and displayed as shown in FIG. 5E. Here, the addresses of the image memory 11 correspond one-to-one with the coordinate record values,
When the recording of the still image by the camera is completed, an individual address (corresponding to each frame) corrected from the total address value of the entire memory is set.

In the above shape extraction, if the extracted shape at the center of the screen is a large shape extending to the edge of the screen, the coordinates of the edge of the screen or the specific area are known in advance. Is detected, a shape of another color is extracted from the extracted shape, and the setting is redone, so that the program is changed so as to obtain a shape that does not cover the edge of the screen or a shape outside the specific area.

Further, in the color setting of the image information, the color information of the extracted image may be detected in advance as described above, and the color detection operation may be performed by a color designation method using the detected data. Note that the rotation angle of the above shape is described from the angle difference in the direction of the longest point or the shortest point from the center of gravity of the tracking shape of the same shape as the reference shape, but is not limited thereto, and , "Image Processing Industry Application Directory"
It may be realized by using a circle element method or a linear element method described on page 409, issued on January 17, 1994, Fujitsu Technosys, Inc.

Here, the circular element method focuses on a local feature such as a hole in a circular shape, and passes through a local feature such as a hole around the center of gravity of the reference circular shape and the tracking circular shape. In this method, a circular line is generated, and the amount of rotation deviation is calculated from the angle of the center of gravity and the change point of a total of two local hole points of the reference circular shape and the local hole point of the tracking circular shape. ,
In the reference shape composed of straight lines, the gradient of each line segment of the reference shape and the tracking shape is determined, and the intersection point of the same line segment is set as the origin, and the gradient is set as the amount of rotation deviation, but the details are omitted.

Next, a third embodiment will be described below with reference to FIG. In the figure, components having the same functions as those in FIGS. 1 to 5 are denoted by the same reference numerals, and description thereof will be omitted.

In FIG. 6, reference numeral 7i denotes tracking shape display means;
Reference numeral j denotes a calculating unit that calculates the rotation angle of the display by the operation of the key 14 or the rotation angle obtained from the gravity sensor and the amount of rotation deviation from the reference shape to control the image direction of the display. .

Although the gravity sensor is not shown, it is needless to say that it may be the same as that described in the conventional example (Japanese Patent Application No. Hei 8-319069).

Reference numeral 7k denotes an image direction correcting means for correcting the rotation of the coordinates on the screen about the center of gravity of the reference shape by the calculating means 7j.

Next, the operation will be described. Before the display surface of the display device is rotated, an erect image is designated, its shape is extracted, and it is made a reference shape in the same manner as described above. Thereafter, when only the photographer's display surface direction is rotated, for example, 90 degrees to the right, the photographer's key 14
An operation or an output from the gravity sensor integrated with the display device is detected, and a right rotation instruction is performed by 90 degrees.

In this case, the photographer operates the key 14 or the gravity sensor is used as image direction setting means for setting a new image direction.

[0104] The signal detected in this way is converted into a RISC-C
It is input to one of the calculation means 7j of the PU 7. The erect image described above is used as a reference shape, and the amount of rotational deviation of the same shape is detected and input to the other of the arithmetic means 7j. The rotation of the tracking shape, which has slightly changed from the reference shape, is corrected by the image direction correction means 7k by changing the rotation direction calculated by the two inputs to the 90-degree right rotation display at the same time, and the erect image is displayed in real time on the color liquid crystal display means. Displayed at 16. Thus, the photographer can always obtain an erect image even if the display surface is rotated. Further, in the image memory 11, since the image is corrected by the image direction correcting means 7k into an image obtained by rotating the display surface by 90 degrees, the image is recorded with that value. Conversely, even if the image is recorded in the image memory 11 on the display surface that has been returned to the original position by 90 degrees, the direction changes depending on the direction of the display surface during reproduction. It is not limited.

When the reference shape is extracted after the rotation of the display surface of the display device, for example, an operation signal of the photographer is operated by the key 14 of the photographer or a gravity sensor to generate a 90-degree right rotation instruction signal. One is input and the image is rotated 90 degrees to the right and displayed upright to the photographer. Thereafter, the reference shape of the erect image in the display is extracted, and the amount of rotation deviation from the reference shape is detected by the same treatment as described above. This is called RISC-C
It is input to the other of the operation means 7j of the PU 7.

Since the input values of the calculating means 7j are the same as before the rotation of the display surface, the same processing as described above is performed, and the photographer can always obtain an erect image even if the display surface is rotated.

As described above, an erect image display is always obtained even when the reference shape is set before and after the instruction to rotate the display surface.

RISC-CP for performing the above image rotation processing
As an example of U7, processing of 93 ns / pixel (4-bit pixel) at a rotation angle of 14 degrees can be considered.

For example, in order to correct a rotation angle of 14 ° with 250,000 pixels / 8 bits, the monthly magazine “Electronic Materials”, 1992
According to the data described on pages 128 to 133 in the April issue, it takes about 46.5 ms. Therefore, this one is TV
However, real-time processing in time for the TV frame period can be performed by using a plurality of these in parallel and performing processing in parallel.

Although the video signal from the imaging device is described by interleaved scanning, the present invention is not limited to this.
Non-interleaved scanning may be used. In this case, it is needless to say that the configuration of the screen can be simplified since the vertical address of the screen can be counted in order.

Although the embodiment has been described above by focusing on minimizing the rotation of the object of interest on the screen as a matter of course, it is needless to say that the present invention is not limited to this mode when performing photographing.

For example, since some objects move in various ways, it is sometimes unclear whether the object has rotated or the camera that shoots has rotated. Therefore, a configuration may be adopted in which a certain degree of rotational direction correction effect is provided, and the correction effect fades over time. Hereinafter, an embodiment according to this specification will be described with reference to FIG.
This will be described with reference to the block diagram of FIG.

In the figure, those having the same functions as those in FIGS. 1 to 3 and FIGS. 7 to 9 described above are denoted by the same reference numerals, and description thereof will be omitted.

The reference coordinate resetting means 7L resets the reference coordinates for another subject and continues tracking when the extracted shape moves outside the specific area. Reference numeral 7m denotes a correction amount adjusting means. When a change in the rotational displacement detected by the first rotational displacement detecting means 7c is larger than a certain value, the correction amount by the first coordinate correcting means 7d is limited to a certain value or less. In the case where the over-correction is prevented, and the movement amount of the extracted shape detected by the reference coordinate resetting means 7L deviates from the center by a certain amount, the first reference specifying means 7a is reset to perform the above-described operation. Reference coordinate resetting means 7
By repeating the resetting more frequently than the resetting frequency originally controlled by “a”, it is possible to suppress the blur in the rotation direction of the subject substantially at the center of the screen.

Of course, the start of resetting may be detected by providing a specific area in addition to the center of the screen, or the correction amount of the rotational deviation may be reduced as approaching the specific area to deviate from the center. Needless to say, in the case of a subject, the correction of rotation may be made weaker.

[0116]

As described above, according to the present invention, an erect image is initially specified in a captured image, the shape of the specified color is extracted in real time, and the extracted shape is determined from the initially set shape. The rotational displacement amount is calculated and detected, and the rotationally displaced image is corrected in real time in the direction of the initially set erect image shape and displayed, so that even when the image rotates at high speed, the erect image can always be displayed. This has the effect of making images easier to see and preventing visual fatigue and dizziness.

Further, since the correction is performed so as to maintain the angle of the shape based on the shape of the erect image initially set as the reference and the display is performed, the same shape following the erect image based on the reference is continuously rotated. Even in this case, since only the background screen rotates and the front direction of the reference shape does not always change, there is an effect that the reference shape can always be monitored and observed with an erect image.

Further, since the erect image direction is corrected in accordance with the switching of the display surface direction of the display unit, the erect image can be always displayed even if only the display direction of the display unit is changed.
In addition, there is an effect that is easy to use.

Further, since the rotation of the new reference shape obtained by moving the photographed image can be set in the direction of the first erect image from the reference shape of the first erect image, the photographing area can be widened.
There is an effect that an erect image can always be displayed during image rotation.

Further, since a series of input images from a VTR, a video camera or the like can be subjected to the same processing as described above, there is an effect that an erect image can always be displayed by rotating the image.

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing circuit blocks of an image display device according to a first embodiment of the present invention.

FIG. 2 is a configuration diagram showing a circuit block in detail of a shape extraction unit according to the first embodiment of the present invention.

FIG. 3 is a view showing display of an image according to the first embodiment of the present invention.

FIG. 4 is a configuration diagram showing circuit blocks of an image display device according to a second embodiment of the present invention.

FIG. 5 is a diagram showing a display of an image according to a second embodiment of the present invention.

FIG. 6 is a configuration diagram showing circuit blocks of an image display device according to a third embodiment of the present invention.

FIG. 7 is a flowchart of a first reference specifying unit according to the first embodiment of the present invention.

FIG. 8 is a flowchart of a first tracking unit and a first rotational deviation amount detecting unit according to the first embodiment of the present invention.

FIG. 9 is a flowchart of a first coordinate correction unit according to the first embodiment of the present invention.

FIG. 10 is a flowchart of a re-designating unit and a second tracking unit according to the second embodiment of the present invention.

FIG. 11 is a flowchart of a second rotational displacement amount detecting unit according to the second embodiment of the present invention.

FIG. 12 is a flowchart of a second coordinate correcting circuit according to the second embodiment of the present invention;

FIG. 13 is a configuration diagram showing blocks of an image display device according to a fourth embodiment of the present invention.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Optical system part, 2 ... Image sensor, 5 ... Camera signal processing part,
6 field memory FM, 7 RISC-CPU, 7
a: first reference designating means, 7b: first tracking means, 7c: first rotational displacement amount detecting means, 7d: first coordinate correcting means, 7e
... reference re-designating means, 7f ... second tracking means, 7g ... second rotational displacement amount detecting means, 7h ... second coordinate correcting means, 7i ... tracking shape displaying means, 7j ... calculating means, 7k ... image direction correcting means, 8 ... shape extraction means, 10 ... memory control, 1
DESCRIPTION OF SYMBOLS 1 ... Image memory, 16 ... Color liquid crystal display means, 21 ... Normalization part, 22 ... Logical sum part A, 23 ... Logical sum part B, 24 ...
Filter parts A, 25 ... Filter part B.

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁶ Identification code FIG06F 15/66 350A 15/70 410

Claims (3)

[Claims] An imaging unit for converting an optical image into an electric signal and outputting image information, or an input / output unit for inputting and outputting a series of image information; A shape extracting means for detecting a designated color in the output image information and the luminance of the designated color and extracting a shape obtained according to the designation in the image; A first reference designating unit that is set as a reference for tracking features such as coordinates, a position of a center of gravity, and a direction, while extracting the same shape as the reference shape designated by the first reference designating unit. A first tracking means for tracking, and a first point for detecting a direction of a specific point or a specific linear portion from a center of gravity of a shape on a screen obtained by the first reference designating means or the first tracking means. Direction detecting means, and the first reference finger A direction detected by the first direction detecting means from a shape extracted by the first tracking means, based on a direction detected by the first direction detecting means from a reference shape specified by the means; First rotational deviation amount detecting means for detecting the rotational deviation amount of the first motor;
The center of gravity of the shape on the screen obtained by the first tracking means, or the intersection of the specific linear portion, or the straight line of the specific linear portion, based on the rotational deviation amount of the rotational deviation amount detecting means. Rotate the coordinates on the screen around the intersection of
First coordinate correcting means for obtaining an output image matched with the direction of the reference shape specified by the first reference specifying means; and a first recording means for recording the output image of the first coordinate correcting means.
An image display device comprising: a recording unit of (1) or a first display unit for displaying. 2. The image display device according to claim 1, wherein the same shape as the reference shape set and extracted by said first reference designating means is assigned to a designated specific area on a screen. When moving, or when moving more than a specified amount, a new reference shape is designated in a location different from the location occupied by the moved shape, and the coordinates and center of gravity of the designated new reference shape are designated. Position, direction, etc. are set as a new reference, and if the same shape as the shape specified as the new reference shape has moved to a specified specific area on the screen, or has moved a specified amount or more. In this case, further, a reference re-designating means for repeating the setting operation for designating the new reference shape, and the same reference shape specified by the first reference designating means and the reference re-designating means. form From the center of gravity of the shape on the screen obtained by the second tracking means, the first reference designating means, the reference re-designating means, and the second tracking means. A second direction detecting means for detecting the direction of the specific linear portion, and a reference shape specified by the first reference specifying means obtained by the second direction detecting means, or the reference re-designating means Based on the direction of
A second rotational shift amount detecting means for detecting a rotational shift amount in the direction of the shape on the screen obtained by the second tracking means;
On the basis of the rotational displacement detected by the second rotational displacement detecting means, the center of gravity of the shape on the screen obtained by the second tracking means, or the intersection of the specific linear portion,
Alternatively, a second correction is performed in which the coordinates on the screen are rotationally corrected about an intersection on the straight line of the specific straight line portion to obtain an output image matched with the direction of the reference shape specified by the first reference specifying means. An image display apparatus comprising: a coordinate correcting unit; and a second recording unit that records an output image of the second coordinate correcting unit, or a second display unit that displays the output image. 3. The image display device according to claim 1, wherein the image display direction setting means sets an image display direction of the first and second display means. Arithmetic means for inputting the output of the image direction setting means, the output of the first rotational deviation amount detecting means or the output of the second rotational deviation amount detecting means, and the output of the image direction setting means to calculate the display direction An image direction correcting means for correcting an image display direction by an output of the calculating means; a third recording means for recording an output image of the image direction correcting means;
An image display device comprising image display means.