JPS63191476A - Intelligence image communication system - Google Patents

Abstract

PURPOSE:To transmit a moving picture by using a line with a low bit rate by describing an object body with radial vectors and a normal vectors and sending them to a reception side, and converting reception-side image data base on the reception side and displaying an object body image. CONSTITUTION:When vectors which terminate at points (body surface point) where respective radial vectors originating from the center of rotation of the body strike on the surface of the body are considered and regarded newly as radial vectors, the surface points of the body are represented as a set of those radial vectors. Further, normals to the body at the respective surface points (radial vector point) of the body are calculated and their direction vectors are stored (in a data base) as the normal vectors together with the radial vectors in pairs. Those radial vectors are directed in three dimensions, so the surface shape of a three-dimensional body can be represented and only the surface points are described, so the amount of necessary data may be small.

Description

Detailed Description of the Invention (1) Technical field to which the invention pertains The present invention applies image recognition and image generation techniques to image communication,
This invention relates to an intelligent image communication system that realizes moving image communication using a low bit rate transmission line.

(2) Conventional technology Video telephone systems and video conference systems are known as communication systems that use video communication technology and allow people to see what is going on at a remote location from the comfort of their own home. but.

In the conventional method, the image on the transmitting side is transmitted directly as an image signal, which has the drawback of requiring a very high bandwidth and transmission bit rate for the signal to be transmitted, which increases the transmission cost. there were.

Furthermore, since no judgment has been made regarding the interpretation of the meaning or content of the image on the sending side, the interpretation of the information is left to the judgment of the receiving side, which is not necessarily desirable from the standpoint of uniform information transmission. There wasn't.

(3) Purpose of the Invention In order to compensate for the above-mentioned drawbacks, the system of the present invention recognizes images on the transmitting side using image recognition and image processing techniques that have recently been significantly advanced, and transmits only the recognition results.

Image communication is realized by generating 5 images using an image database on the receiving side, and it is expected to significantly reduce the transmission bit rate, as well as unify the interpretation of 1 image and improve the flexibility of 9 image expressions. It is possible.

(4) Structure of the invention FIG. 1 is a block diagram showing the structure of an embodiment of the invention.
1 is the subject to be sent, 2 is the TV camera source 3 is the image recognition device, 4 is the sending side image data storage device, 5 is the signal transmission device, 6 is the transmission path, 7 is the signal receiving device, 8 is the image generation device, 9 is In the receiving side image database device, 10 is an image display device.

Next, an outline of the operation of the intelligent image communication system of the present invention will be explained using FIG.

First, a subject 1 to be sent is photographed by a television camera 2 and inputted to an image recognition device 3 as television image information. On the other hand, in the transmitting side image database device 4, there is various information 1 regarding the subject 1, that is, a standard image of the subject 1, and deformed images resulting from various operations.

Meaning of a deformed image For example, when the subject is a person, information such as the facial expression of the person implied by the deformed image is stored as an image database.

The image recognition device 3 is a device mainly based on a microprocessor, which has recently made remarkable progress in terms of technology, and is capable of processing the input television image information and an image database related to the subject I stored in the image database 4 on the transmitting side. By comparing and performing matching processing, the shape and movement of the subject, the facial expressions of the person, etc. are recognized. This recognition process for the object is realized using computer-based feature extraction processing and pattern matching processing technology, which are commonly known as image processing technology.

If the system is configured to perform processing such as identification of two people and recognition of movements as recognition processing for this target person, the recognition results will be transmitted to the receiving side to display the attendees and the speaker's identity. I said display. It is now possible to make decisions in a unified manner, which in the past had to be determined by the receiver by looking at the received image.

The recognition results for these objects are sent to the transmission line 6 via the signal transmission device 5 as object movement commands and facial expression commands. Here, the signal transmission device 5 converts various commands that are the recognition results of the image recognition device 3 into transmission signals, and also performs transmission transmission, signal transmission and reception control, error correction, etc. with the signal receiving device 7 on the receiving side. Execute transmission control procedures.

When the signal receiving device 7 on the receiving side receives the transmission signal from the transmitting side, it converts it into various commands and sends them to the two-image generating device 8.

The receiving side image database device 9 also contains various information regarding the subject 1 similar to that of the transmitting side image database device 4, including standard images of the subject 1, deformed images resulting from various operations,
Meaning of a deformed image For example, when the subject is a person, information such as the facial expression of the person implied by the deformed image is stored as a two-image database.

Therefore, the 9-image generating device 8 uses the image database in the 2-image database device 9 based on various commands sent from the signal receiving device 7 to generate a facial expression image accompanied by the movement of the subject 1. Send to. Image display device 10 displays image information in real time. As a result, the image on the transmitting side is recognized, transmitted to the receiving side at a low bit rate, and is reproduced and displayed.

FIG. 2 is a diagram illustrating a mode of extracting the movement of a target person. FIGS.
1 and 103 are the right eye patterns in the image frame, respectively;
102 and 104 are left eye patterns.

Figure 3 is a diagram explaining how to calculate the rotation angle of the head.
This shows the head viewed from above.

α is the angle between the center of rotation and the front direction of the eye.

β is the rotation angle of the head.

Here, the movement of two people is generally extracted and recognized by dividing into parallel movement and rotation around a certain point. first,
The manner in which the rotation angle of the head is determined will be explained using FIGS. 2 and 3.

An image frame containing the target person (Figure 2)
a)) and the next image frame (FIG. 2(b)), an eye extraction pattern, for example 101, is set at the position of the right eye of the person in the second image frame (FIG. 2(a)). . Next, apply this cutting pattern to the next image frame (second
Moving to Figure (b)), we also match this pattern with the right eye.

The position where this matching is achieved becomes the apparent movement position of the right eye. In the same way, the apparent moving position of the left eye can be determined.

Here, assuming that the viewpoint is at infinity, the apparent distance between the two eyes (dZ in FIG. 3) can be found by the above operation. Therefore, if the distance between the eyes of the target person when facing the front (dI in FIG. 3) is known in advance, the rotation angle β of the person's head can be determined using the following equation. That is.

d, = 2r sin α (L) dz
−Tsin(α1β)+rsin(α−β)02γSi
Since nαCO3β, β-arccos (dz /d+)
(2) The vertical rotation angle of the head can also be determined in the same manner as above using the eyes and mouth as feature points.

It is clear that the motion of parallel movement of a person can be extracted using a pattern matching method similar to that described above using points such as the tops of heads and chins of two people.

As a result, on the transmitting side, the movement of the person can be recognized and extracted as parallel movement and rotation around a certain point.

FIG. 4 is a diagram illustrating the three-dimensional expression mode of object images such as human face images in the 9-image database, and 400 is an object such as a human head that is a recognition target defined in a three-dimensional space; 401 is the origin of the three-dimensional space taken at the center of rotation of the object, 402 is a radiation vector radiated from the origin in each direction up and down, left and right, two forwards and backwards, and 403 is a reference cube that defines the radiation vector.

At this time, the radiation vector is constructed as follows.

That is, each face of the reference cube is nXn (for example, n=200
(x200, etc.) into square lattices. Then, a vector (radiation vector) connecting the center of the reference cube and each of these lattice points is created.

Using this radiation vector, a three-dimensional object can be expressed as follows. In other words, for each radiation vector radiated from the center of rotation of an object, if we consider a vector whose end point is the point where it hits the object surface (object surface point), and we call this a radiation vector, we can calculate the A point can be represented by a set of these radiation vectors. In addition, in order to display the object image at high speed, the normal line of the object at each surface point (radiation vector point) of the object is calculated, and the direction vector is stored as a normal vector and paired with the radial vector (database). do.

As can be seen from the way it is constructed, this radiation vector is radiated in all directions in three-dimensional space, so it is possible to express the surface shape of a three-dimensional object, and since only surface points are described, the amount of data required is reduced. Less is better. Furthermore, as described below, it has the feature that changes in the position of an object due to rotation or movement of the object can be easily calculated.

That is, when the object moves, the database can be changed as follows. In general, the movement of an object can be divided into two rotations and a parallel movement, as described above. Also here 9
The movement of objects to be communicated, such as people, is separated into rotation and translation for extraction and recognition.

As the object rotates, each surface point of the object also rotates by the corresponding angle, so it is sufficient to rotate the radiation vector by the corresponding angle.

At this time, the normals of the corresponding surface points are also rotated by the same angle, so it is only necessary to rotate the normal vectors by that amount.

Furthermore, for parallel movement, the radiation vector may be shifted by a corresponding amount.

Processing such as rotation and movement is arithmetic processing of numerical data stored as a database.

It is easily realized using ordinary processor technology.

In this way, when two objects are described by a radiation vector and a normal vector, the line of sight vector is used to correspond to the image after movement, as in the ray tracing method and scan line method, which are known as conventional image processing methods. Since there is no need to calculate the intersection between the object and the object, and only vector rotation and movement calculations are required, it is possible to quickly transform and create a database representing the image of the object after it has moved.

FIG. 5 is a diagram showing the structure of the object image database in the database, where 500 is an object name command such as a person's name;
01 is a partial object name command such as a head, chest, or arm, 502 is a deformed object part command that partially deforms two eyes, etc., 503 is a normal vector command, and 504 is a radiation vector command. Here, the two-part object has a deformed object part, and
Please note that there are some that do not have a deformable object part.

The transformation of facial expressions when a person smiles or gets angry cannot be handled simply by rotating or moving the radiation vector, so the database is transformed as follows. In other words, the database of two human figures is divided in advance into two parts, such as shoulders and chests, which only move as the two bodies move as a whole, and parts that change their shape, such as eyes and mouths, as shown in Figure 5. They are written separately. Also, at this time, a plurality of shape data regarding the deformed object portion are stored in parallel. Therefore, when the transmitting side recognizes which of the deformed object parts of the target object is the deformed object part, the data (radiation vector and normal vector) of the deformed object part that is most suitable for that deformation is selected from among them. . This identification information is also transmitted to the receiving side as transmission information.

The receiving side extracts the corresponding deformed object part data based on the transmitted information and inserts it into the corresponding part of the database to assemble the database that becomes the basis for generating one image.

Thereafter, deformation of the database based on the movement of the entire object is performed using this database as described above.

In this way, the object to be transmitted on the transmitting side, including its motion, is recognized on the transmitting side, and the recognition result is transmitted to the receiving side. On the receiving side, the image of the target object can be reconstructed from this recognition result, so the receiver can know the status of the transmitting side by looking at the image.

(5) Effects of the Invention As described above, according to the present invention, since an object is described by a radiation vector and a normal vector, it is possible to generate images in real time, and it is also possible to generate images in real time. On the other hand, the data to be transmitted is only the rotation of the object, the amount of parallel movement, and the identification information of the deformed part, and the amount of transmitted information is small. In other words, by recognizing and recognizing the movement of the target object, it becomes possible to greatly compress and transmit the information to be transmitted.
It becomes possible to communicate two moving images using a line with a very low bit rate.

In addition, since the image recognition on the sending side recognizes the facial expressions and actions of the target person, if the recognition results are transmitted to the receiving side and an appropriate display is performed, the receiving side can uniformly interpret the intentions and conditions of the sending side. It can be transmitted to.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention, FIG. 2 is a diagram explaining a mode of extracting the movement of a target person, and FIG. 3 is a diagram explaining a mode of calculating the rotation angle of the head. Figure, 4th
The figure is a diagram for explaining the three-dimensional representation of the target object in the image database, and FIG. 5 is a diagram showing the structure of the object image database in the database. DESCRIPTION OF SYMBOLS 1... Subject, 2... Television camera, 3... Image recognition device, 4... Transmission side image database device, 5...
... Signal transmission device, 6... Transmission path, 7... Signal receiving device, 8... Image generating device, 9... Receiving side image data storage. 100... Target person image, 101, 103... Right eye pattern, 102, 104... Left eye pattern, d, d
2... Eye spacing, 400... Object placed in three-dimensional space, 401... Origin of three-dimensional space, 402...
Radiation vector, 403...Reference cube, 500...
Object name command, 501...Partial object name command, 5
02... Deformed object part command, 503... Normal vector command, 504... Radiation vector command.

Claims (3)

[Claims] (1) In an image communication system that transmits an image corresponding to a subject, there is a transmitting side device consisting of a television camera device, an image recognition device, a transmitting side image database device, and a signal transmission device, a signal receiving device, an image generating device, and a receiving side. An image database device and a receiving device consisting of an image display device are connected by a transmission path, and in both the transmitting and receiving databases, a radiation vector from the origin taken in the three-dimensional space in which the target object exists is stored. and normal vector,
An intelligent image characterized by recognizing a target object by an image recognition device on the transmitting side, transmitting the recognition result to the receiving side, and based on the recognition result, converting the receiving side's image database on the receiving side and displaying the image of the target object. Communication method. (2) In the intelligent image communication method according to claim (1) above, the image recognition device on the transmitting side recognizes the movement of the target person as recognition of the target object. method. (3) An intelligent image communication system according to claim (1), characterized in that a target person is identified as recognition of a target object by an image recognition device on the transmitting side.