JPS63276672A - Intelligent work station - Google Patents

Abstract

PURPOSE:To effectively understand contents and to form picture data which can be supplemented by analyzing a character string between instructed sentence blocks, extracting sentence information between the sentence blocks, forming and displaying an animation picture according to the extracted sentence information. CONSTITUTION:At the time of the input of sentence data or the designation of the sentence data (A), the character string of the sentence data is displayed on a display (B). At the time of designating (C) the sentence punctuation for forming the animation picture to the displayed sentence data, the character string between the sentence punctuation is analyzed to extract the sentence information required for forming the animation according to the extracted sentence information (D, E). Then, according to the extracted sentence information, the animation is formed (F), the picture is displayed (G), the data is associated with the sentence data and stored. Thereby, the element of a display graphic supplied from a data base varied according to the sentence information, numerical data or a time relation thereof and the animation picture in which the contents of the sentence data can be visually impressed is formed.

Description

[Detailed Description of the Invention] [Object of the Invention] (Field of Industrial Application) The present invention relates to an intelligent workstation that facilitates understanding of the contents of output text data.

(Conventional technology) One of the important primary processing capabilities of a workstation is
There is a presentation of various text data. For example, the text data may be displayed in two lines using a display device, or the text data may be output as audible text using a speech synthesis function.

By the way, even if sentences f and f data, such as explanatory sentences, are presented from a workstation, the contents, specifically the numerical data indicated by the contents of the sentences, may not be easily understood. In such cases, it is common practice to display the numerical data in a graph (diagrammatic form) to assist in understanding the content.

However, the creation of graphical data from text data in conventional systems generally only creates graphs and comparison tables as still images. For this reason, it is extremely insufficient to easily understand, for example, the state of change in numerical values over time.

(Problem to be solved by the invention) In this way, in conventional workstations, graphical information such as graphs to help the understanding of text data is analyzed and created as a still image by analyzing the text data. of things,
There have been problems such as the fact that understanding the content of the text data may not be sufficiently assisted only from the still image.

The present invention has been made in consideration of these circumstances, and its purpose is to provide knowledge that can analyze text data and create image data that can effectively assist in understanding the content. The aim is to provide a workstation with a wide range of functions.

[Structure of the Invention] The present invention, as shown in the concept in FIG.
, displays the character string of the text data on a display (processing B). When a text section for creating a video is specified for this display text data (processing C), the text information necessary to create a video is extracted by analyzing the character strings of the text section (processing ri, E). Then, a moving image is created according to the extracted text information (process F), the moving image is displayed (process G), and the moving image data is stored in association with the text data.

In other words, the character strings in the designated specific sections of text data are analyzed and the text information is extracted, for example, the subject or subject.

Numerical data, temporal relationships, etc. are extracted and, for example, the elements of display figures given from the database are processed variably according to the above text information, especially according to the numerical data and their temporal relationships. The purpose is to create a video that can give a visual impression of the content.

(Operation) According to the present invention, various numerical data and information relating to subjects and temporal relationships, which associate these numerical data, are extracted from text information obtained by analyzing text data.

Then, based on this information, the graphic elements provided from the database are transformed, and, for example, a moving image expressing the temporal transition of numerical data is created and displayed.

Therefore, according to this video, it is possible to easily present information such as whether the numerical data has changed rapidly or slowly, and can extremely effectively help the user understand the content of the text data. It becomes possible. In other words, unlike static images that show fixed changes in numerical data,
Since the changes in the numerical data can be expressed as changes over time using a video, it is possible to accurately represent the contents of the text data and effectively aid in understanding the contents.

(Example) Embodiments of the present invention will be described below with reference to the drawings.

FIG. 2 is a schematic diagram of an intelligent workstation according to an embodiment of the present invention. This intelligent workstation
It is composed of the following parts.

Bus 1: Used to transfer necessary information between each section described below.

Control unit 2: Mainly composed of a microprocessor, it controls the operations of each part of the intelligent workstation.

Image input device 3; camera, scanner, or OC
R, etc., and input various image information.

Position coordinate input device 4: Consists of a tablet, mouse, etc., and inputs specified position coordinate information.

Audio input unit 5: Consists of a microphone, etc., and inputs audio information.

Keyboard section 6: Equipped with a plurality of keys, for inputting character/symbol codes, control codes, etc.

IC card section 7: As described later, an IC card is installed, and necessary information is outputted between the IC card and the IC card.

Bus controller 8; controls information transfer between each unit via bus 1.

Audio output unit 9: Consists of a speaker, etc., and outputs audio information.

Display unit 10: Consists of a CRT display, liquid crystal display, etc., and displays characters, figures, images, etc.

Image output unit ff1ll: Consists of a FAX, color printer, etc., and prints out various image information.

communication device 12.13; the workstation and the telephone;
Or, it communicates information with other workstations, terminals, etc. installed in remote locations. .

Switching device 14: Switches and uses a plurality of communication devices.

Timer part 15: Provides time information and time information to the workstation.

Encryption processing unit 167 encrypts various information.

The voice verification unit 17 performs a verification process to determine whether the given voice information is a specific voice.

Image verification unit 18: Performs verification processing to determine whether the given image information is a specific image.

Speech recognition unit 19. Recognize and process the given audio information.

Speech analysis unit 20: Analyzes the voice input from the voice input unit 5 etc. by extracting the characteristics of the voice.

Character recognition unit 2L: Recognizes character/symbol patterns input from the image input device 3 or the like.

Image recognition unit 23: Recognizes graphic images and the like input from the image input device 3 and the like.

Output format selection unit 24: Selects and controls the format of information output from the workstation.

Work environment data collection unit 25: Collects and inputs information such as the functional status of the workstation and the work environment in the office.

Speech synthesis unit 26: Generates synthesized speech according to the processed data.

Image compositing unit 27; performs compositing processing on a plurality of pieces of image information, and executes image editing processing according to processing data.

Graphic composition processing unit 28: Composes various graphics and executes editing processing such as adding and deleting graphics according to processing data.

Audio compression/expansion unit 29: compresses and encodes audio data, and restores and expands compressed audio data.

Image compression/expansion unit 30; compresses and encodes image data, and restores and expands compressed image data.

Signal processing unit 31: Executes a series of signal processing such as encoding and compressing various signal information, restoring and expanding it, and adding necessary information.

Database unit 32: Classifies various information into a plurality of relations and stores them as a database. still,
This database is constructed not only as code information but also as images, sounds, etc.

91 The intelligent workstation according to the present invention is basically configured with each of the above-mentioned parts, and each of the above-mentioned parts effectively utilizes its respective functions to exhibit an intelligent function as a whole. ing.

Next, the IC card section 7, encryption processing section 16, etc., which are not common like the aforementioned keyboard section 5, etc., but have characteristic functions in this intelligent workstation, will be explained in more detail.

First, an IC card has a semiconductor circuit such as a microprocessor and a memory circuit built into the card body 7a, which is about the size of a business card, as shown in FIG. It is constructed by providing an interface section 7b for connection to the station main body and a display window section 7c.

The display window section 7c is formed by embedding a transparent polarizer, and its position is set so as not to overlap with the interface section 7b or the semiconductor circuit. Also, the card body 7a
Only the portion corresponding to the display window 7c may be transparent, or the entire substrate may be transparent.

Specifically, as shown in an exploded perspective view of FIG. 4, the IC card has a pair of cover substrates 7d.

7e, a buried substrate 7r1, a core sheet material 7g1, a printed circuit board 7h, sandwiched between these cover substrates 7d and 7e.
It is constructed by integrally bonding them together by thermocompression.

An input/output terminal 71 is provided on the printed circuit board 7h at a position facing the interface portion 7b, and a liquid crystal display device 7j is provided at a position facing the display window portion 7c. Furthermore, the semiconductor integrated circuit 7 is mounted on the printed circuit board 71].
k is provided. Further, the cover substrate 7e is provided with a metal foil 7m for dissipating heat generated in the printed circuit board 7h.

Note that the holes drilled in the cover substrates 7d and 7e, the embedded substrate 7r, and the core sheet material 7g are the printed circuit board 7.
hi: Provided at positions facing the integrated semiconductor integrated circuits 7j and the like. The semiconductor integrated circuit 7 described in - is fitted into these holes, and the cover substrate 7 is inserted.
d, 7e, embedded substrate 7'', core sheet material 7g1 printed circuit board 7h are laminated and integrated to form an IC card.The input/output terminals 71 are connected through holes made in the cover substrate 7d. An interface section 7b is exposed and electrically connected to the workstation main body.

The liquid crystal display device 7j has a liquid crystal layer sandwiched between a pair of polyether sulfone film substrates provided with a spacer in between, as shown in FIG. , a transparent conductive film is formed on the inner surface of the film substrate, and a polarizer or a reflector is provided on the lower film substrate. If the liquid crystal display device 7j is constructed using a polyether sulfone film substrate in this way, it is easy to reduce the thickness to 0.6ρ or less, compared to the case where the liquid crystal display device is constructed using a glass substrate. This allows the IC card itself to be made thinner.

Further, the driving power for this IC card may be supplied from the workstation main body via the interface section 7b, or it may be built into the card. In this case, it may be incorporated as a sheet-like battery using, for example, a polymer film.

The semiconductor integrated circuit 7 includes, for example, a CPU 7p and a PROM which is a data memory, as shown in FIG.
7q, E2PROM 7r, and a selection section 78 for these memories.

The PROM 7q is a large-capacity nonvolatile memory that cannot be erased or rewritten, and stores control programs for the CPU 7p, information to be permanently recorded, and the like. Also E2P
The ROM 7r is a rewritable, small-capacity, nonvolatile memory, and stores, for example, information transaction numbers, numbers used in information transactions, and information updated when the information is used.

These memories are selectively driven by Il control of the selection section 7s and input/output information to/from the CPU 7p. The CPU 7p executes necessary information processing using these memories, and also inputs information to and from the intelligent workstation main body via the aforementioned terminal section 71 from its interface section.

The IC card unit 7 is equipped with such an IC card and inputs and outputs information to and from the IC card.

Incidentally, it goes without saying that the IC card is not limited to the configuration connected to -, and it goes without saying that the IC card 1 section 7 can be configured depending on the configuration.

Next, the encryption processing section 16 will be explained.

The encryption processing section 6 includes, for example, an encryption section 16a, a decryption section 16b, and a private key file section 16, as shown in FIG.
c, a public key file section 16d5, and a key update section lee.

As the concept is shown in Figure 8, a given original communication text can be encrypted according to an encryption key to generate the encrypted message, or conversely, a given encrypted communication can be decrypted according to the encryption key to generate the encrypted message. Execute the process to obtain the original text.

Private key file section tCC and public key file section led
stores the keys used for this encryption/decryption, and the key update unit 180 is in charge of updating these files.

Here, the private key is a key known only to the workstation that owns this encryption processing unit 16, and is kept secret from other workstations.

On the other hand, the public key is paired with each private key set in each workstation, and is given to each other workstation and made public. The public key file section led stores the public keys published by these plurality of workstations in correspondence with each workstation.

As shown in FIG. 9, the encryption unit 1[ia is comprised of an R8A processing unit 16i and an encryption type addition unit 16j. When attempting to encrypt the original communication text and communicate information, the original communication text is encrypted using the public key published by the workstation of the communication partner, and information indicating the type of encryption is added to the encrypted communication text. It creates communication information and communicates it. The encryption type information includes, for example, "0" indicating that the encryption is not performed, "1" indicating that the encryption is encrypted, information indicating the encryption method, and the like.

In addition, the decryption unit 16b inputs the encrypted message that has been encrypted and communicated from the workstation using the public key published by the self-workstation, and decrypts it using the private key corresponding to the private key. The first
As shown in FIG. 0, the ciphertext division section 16 includes a cipher type determination section 16111% switching section 16n.

16p and an R3A processing section 16q.

The ciphertext division section 16 is for dividing the communication information communicated in the above-described format into the above-mentioned cipher type information and the encrypted message, and includes a cipher type determination section 16In.
determines whether the message is encrypted or not based on the encryption type information. If the communication is not encrypted, the communication is outputted via the switching unit 1[in, IBp, and if it is encrypted, the communication is guided to the R8A processing unit 16q. The R3A processing section 18q decrypts the encrypted message using the secret key, and outputs it via the switching section 1[ip.

Note that the R3A processing unit IBi, 16q is configured to include a block division unit IBs, an exponentiation/remainder calculation unit 16t, and a block connection unit 16u, as shown in FIG. 11, for example.

Here, the block division section 18s divides the given signal sequence into blocks M of a constant length, and the exponentiation/remainder calculation section 111it divides each block M.

At each time, a signal sequence N, where N,=M,k (mod n), is obtained using an encryption key k. However, n is a fixed value. This signal sequence N is sequentially connected via the block connecting section 16u and output.

In the encryption process, the signal sequence M is the original communication text, and the encrypted communication text is determined from this original communication text as the signal sequence N. In addition, in the decryption process, the signal sequence M is an encrypted message, and the decrypted original communication or signal sequence N is obtained from this encrypted message.

The key k responsible for such encryption/decryption is the aforementioned public key and private key, which are set 17 as a pair.

Therefore, a workstation can each encrypt communication information according to a public key published by another workstation, or can decrypt the encrypted communication only by using the secret paired with the public key. Only certain workstations can know the key. Therefore, a workstation that wishes to encrypt and communicate information encrypts the original communication text according to the public key made public by the workstation with which it communicates. The communication information can only be decrypted by the communication partner's workstation that has the private key.

Note that it is not necessary to store all the public keys published by other workstations in the public key file 1[id. For example, public keys published by each workstation are stored in a file corresponding to each workstation in a public key file memory provided separately for the system. When information communication becomes necessary, the public key of the communication partner may be read from the public key file memory and stored in the public key file section 16 of the own workstation.

The above is the basic configuration of the encryption processing section 16 and its functions.

Next, the image matching section 18 will be explained.

The image matching unit 18 receives image information inputted from the image input device 3, for example, an image of an individual's face, and performs individual identification.

FIG. 12 shows a schematic configuration of this image matching section, in which 18a is an image storage section, 18b is a normalization circuit, and 18a is an image storage section;
c is a binarization (thinning) circuit, and 18d is a feature data extraction circuit. Further, 18e is a data storage section that stores image data, 18f is a search circuit, 18g is a collation circuit, and 18h is an output section.

The image storage section 18a stores image information input through the image input device 3, and performs image verification processing. The normalization circuit 18b performs normalization processing on the image information stored in the image storage section 18a, and the binarization circuit 18c performs binarization processing on the image information.

Specifically, here, the normalization circuit 18b normalizes the size of the individual's face in order to identify the individual based on the image of the individual's face. The binarization circuit 18c performs, for example, edge line detection and thinning processing on the normalized face image to obtain a binary image of the image.

The feature data extraction circuit 18d performs normalization and
This method extracts feature data from binarized image information. That is, in the matching process using a face image, for example, as shown in FIG. 13, the contour of the face is extracted as one feature, and then features such as the eyes, nose, and mouth in that image are also extracted. . Specifically, the contour features of the face are extracted as classified code information, and the distance between the eyes, the size of the mouth (m5), the distance between the eyes and the mouth (n), etc. are extracted as numerical data as features of the image. ing.

Therefore, the data storage unit 18e stores feature data of facial images obtained in advance for each individual, for example, the 14th
It is registered as shown in the figure. That is, the facial image characteristic data described above is registered for each individual using the individual's name as an identification name, and the facial image data are connected by pointers.

The search circuit 18r searches the data storage section 18e based on the feature data extracted by the feature data extraction circuit 18d. The search data is then given to a matching circuit 18g, where it is matched with the feature data obtained by the feature data extraction circuit 18d.

This matching process is carried out, for example, by the feature data extraction circuit 18d.
When the feature data of the input image obtained in is X (1 is the type of feature) and the feature data of the image registered in the data storage unit 18e is Y, then D-Σ IX, -Y, 1. = 21- is performed by performing a calculation and identifying the person with the smallest value as the calculation result. This identification result is output via the output section 18h.

The image matching unit 18 basically performs matching processing on the input image in this manner, and performs, for example, individual identification of the human-powered image.

Next, the speech recognition section 19 will be explained.

The speech recognition unit 19 is configured as shown in FIG. 15, for example. The audio input circuit 19a inputs the audio signal input from the audio input section 5 or the audio signal received by the communication device 12.13 via the public telephone line, and converts this input audio signal into an appropriate form. Amplifiers that amplify the signal to a certain level, bandpass filters that limit the bandwidth, and A/
It is composed of a D converter and the like. The input audio is limited to signals in the frequency band of 30 to 3400 Hz, for example, and is quantized into a 12-bit digital signal at a sampling period of 12 KHz in the audio input circuit 19a.

The acoustic processing section 19b is composed of, for example, a product-sum circuit constructed from dedicated hardware. Basically, it operates at high speed in a pipeline manner in synchronization with the audio input circuit 19a.

The acoustic processing here is performed by two types of bandpass filter groups. One of them is a 16-channel filter bank, through which changes in the spectrum of the input audio signal are extracted.

The other is a gloss filter that divides the same band into four channels, and the acoustic features of the input voice are extracted through this gloss filter.

These two types of filter groups (filter bank and gross filter) are configured, for example, as fourth-order cyclic digital filters. For example, the filtering output is obtained every 10 m5ec. still,
Control of this sound processing section is performed using a microprogram method.

Therefore, the preprocessing/recognition unit 19c is a high-speed processor 19.
d, a pattern matching processing section 19e1, a word dictionary memory 19f1, and a buffer memory 19g.

The buffer memory 19g inputs the audio signal filtered by the audio processing section 19b, and for example, a maximum of 1
．． It stores 8 seconds worth of audio data. The high-speed processor 19d performs voice section detection, resampling, labeling, recognition processing using a transition network, and comprehensive logical judgment processing on the data stored in the buffer memory 19g. The high-speed processor 19d also performs communication with the host computer and controls the overall operation of the speech recognition section 19.

The pattern matching processing unit 19e performs matching processing such as composite similarity calculation on the audio data processed by the high-speed processor 19d with the standard pattern data of the word audio registered in the word dictionary memory 19f,
We are looking for recognition candidates.

For example, speech words to be recognized are uttered discretely.

Therefore, the high-speed processor 19d detects the input section of word speech using, for example, human voice energy calculated every lomsec during acoustic processing.

Specifically, as shown in FIG. 16, the threshold E is adaptively calculated from the background noise level and the input voice level. When the input audio signal level exceeds the threshold value Eo for a certain period of time or more, the time point at which the input audio signal level exceeds the threshold value Eo is detected as the start point S of the audio word. Thereafter, when the level of the input audio signal falls below the threshold E for a certain period of time or more, the end point E of the audio word is detected as the point in time when it falls below the threshold Ee.

By the way, speech recognition can be considered as a type of pattern recognition. However, there are variations in the patterns peculiar to speech, individual differences due to the gender of the speaker, the shape of the vocal organs, the method of enunciation, etc., as well as noise generated by the speaker himself or herself, noise from the surrounding environment, and even, in the case of telephone speech, public noise. There are problems with level differences and noise caused by going through a telephone line. Therefore, the problem is how to accurately and stably recognize speech by taking these into consideration and absorbing the above-mentioned variable factors.

Therefore, the preprocessing/recognition section 19c employs a recognition method called hybrid structure matching method, which combines pattern matching method and structure analysis method in two stages.

That is, when a word voice section is detected as described above, the first voice section (S, E) is divided into 15 equal parts, and each of the 16 points is set as a resample point. Then, the spectrum at each resample point is extracted from the 16 channels of audio data (spectral time series) that have been acoustically processed as described above. Note that if there is a deviation between the sample point of the audio data and the resample point, it is better to extract the spectrum of the nearest point to the resample point.

By this resampling process, for example, 113XlB (~2
56) Find the dimensional audio pattern vector X.

That is, when the j-th (j = 1.2, 3. to 16) sample point is r, the vector data of 16 channels at r is s, = cs, s, ~S) r JlrJ
, 2rJ, 16r, +teX=(S S-3-8)' lrl, 1r2. 2rl, 1
Find the vector X of the voice pattern [irl[i.

However, t indicates the transposition of the matrix.

Human speech pattern vector X obtained in this way
The degree of similarity between this word and a standard pattern of word sounds registered in advance in the word dictionary memory 19r is calculated by, for example, a composite similarity method.

Here, the standard pattern of word sounds registered in advance in the word dictionary memory 19f is (ψ ψ ~ ψLk) 1 to 2k for the word category ωk.

(λ λ ~ λ ) 1, 2, Lk However, (λ ≧λ ≧ ~ ≧ λLk) 1 and -2 are prepared. In addition, ψ λ is the category,
, the eigenvectors and their eigenvalues in the covariance matrix K of the pattern hector X belonging to ωk. For such a word dictionary, the above-mentioned composite similarity S (k) is calculated as follows. Note that in the above equation, l X II is the norm of the vector X.

Such composite similarity calculations are performed for all categories, and the top similarity value and the resulting category name are determined as a pair.

Pattern matching using such a composite similarity method enables recognition processing that eliminates many pattern variations. However, for similar patterns or patterns with added noise, the difference in similarity values between different categories may become small.

Therefore, as mentioned above, the following structural analysis method has been introduced as a supplement to the pattern matching method. This structural analysis performs recognition processing by focusing on the differences in the sounds that make up word speech, and consists of two types: a phoneme label series and an acoustic feature series.
It uses two time series.

That is, the phoneme label sequence is 10m5e from the input audio signal.
The degree of similarity with the phoneme dictionary is calculated using the spectrum of 16 channels calculated for each c, and phonemes having a degree of similarity of a certain value or more are labeled and determined. This phoneme label is
For example, it consists of six types: five vowels and nasal sounds. On this occasion,
It is better to prepare separate phoneme dictionaries for male and female voices.

Compared to vowels, which are pronounced relatively stably, it is difficult to label consonants individually as phonemes. Therefore, the acoustic characteristics of the consonant are labeled and used as characteristic information.

Specifically, acoustic features are extracted from the output of a four-channel gross filter and audio energy obtained through acoustic processing. The acoustic features extracted and labeled in this way include, for example, silence, voicelessness, friction, rupture, energy, etc., as shown in FIG. 17 in comparison with the output features of the gross filter. It consists of 12 types including dips.

Thus, the phoneme/acoustic label sequence obtained for the input speech is as follows over the range including the speech period (S, E):
The information is input to a transition network created for each word category, as shown in FIG. 18, for example.

1. Check whether the specified phoneme label or acoustic feature exists for each node in this transition network. Then, if there is no characteristic, it is rejected, and if it exists, it is transferred to the next node, and when that characteristic sequence is completed, the input sequence that has reached the goal of the transition network is accepted, and its category is determined. Note that the direction of sequence checking can be selected from forward to reverse for each network.

The comprehensive judgment logic is a logic that synthesizes the candidate categories ordered by pattern mapping as described above and the recognition results obtained by the transition network, and makes a final judgment.

That is, in this comprehensive judgment logic, when the maximum similarity obtained by pattern mapping is set to 81, this is compared with a predetermined threshold value θ. In the case of (Sl×θ), this is rejected as noise.

In addition, in the case of (S1≧θ), another threshold value Δθ is used to
Categories with a degree of similarity greater than or equal to S, -Δθ) are extracted as candidates. If the number n of extracted categories is one, this is extracted as the recognition result. Furthermore, when a plurality of categories are extracted, only the categories accepted by the transition network are extracted by referring to the analysis results by the transition network. Then, the category with the greatest degree of similarity is determined as the recognition result.

In addition, if the categories extracted by threshold processing do not include those accepted by the transition network,
It is assumed that it is impossible to judge.

As described above, the input word speech is recognized by integrating the pattern recognition processing result using the composite similarity method and the recognition result using the transition network.

Figure 19 shows the flow of the word speech recognition processing procedure in this speech recognition unit. After speech segment detection processing, resampling processing is performed to perform pattern matching, and at the same time, labeling processing is performed to check using a transition network. After that, these recognition results are integrated to perform comprehensive judgment logic processing. Such processing is executed under a processing sequence by the high-speed processor 19d.

By the way, it is not a discretely uttered word sound, but 31
If you want to recognize words in one continuous voice, you can do the following. That is, in this case, the input speech may be divided into various sub-intervals, and the word similarity may be determined by performing word identification for each sub-interval.

Specifically, for example, as shown in FIG. 20, all analysis frames in an input speech section are set as boundary candidates for partial sections, and the input speech section is divided into a plurality of subintervals. At this time, since the maximum time length D and the minimum time length D can be set for the duration of the word to be recognized, only a partial interval 1n within that range needs to be targeted for recognition processing.

In the example shown in FIG. 20, two partial sections are calculated assuming that the number of words in the continuously uttered voice is two. However, since the number of words in the input speech is generally unknown, it is better to detect each partial interval on the assumption that 2 to n words exist as word candidates. Then, calculate the word similarity for each detected subinterval,
The connection relationships of the similarity results may be compared with each other to find the most reliable boundary of the sub-interval, and then the word recognition results for each of the sub-intervals separated by the boundary may be obtained.

However, when calculating word similarity by obtaining subintervals in this manner, the number of subintervals becomes enormous, which hinders speeding up of processing. Therefore, in practice, in consideration of speeding up the processing, for example, if the number of input words is 2 = 5 words, 1
The word duration length is 128 to 640 m5ec, the ratio of word length in one utterance is 2.5 or less, and the frame period is IEimsec (one word out of every two words is extracted in an 8m5ec period). It is good to detect subintervals.

In this way, it becomes possible to effectively recognize each word in continuously uttered speech.

By the way, learning of a dictionary (word dictionary) used for such speech recognition processing is performed as follows.

This learning process is roughly divided into (1) a process for finding the characteristic kernel from the vowel pattern and the consonant pattern, and (2) a process for finding the eigenvalue and eigenvector for the characteristic kernel. Then, these eigenvalues and eigenvalue vectors are divided into N parts in descending order of the eigenvalues. This process is generally called KL expansion.

First, the process of determining the characteristic kernel will be described. The characteristic kernel of an input speech pattern (learning pattern) is determined as follows, where S is the vertical vector of the learning pattern.

K = (17M) Σ ss' tll-1m m 5=(S S -8)' m ml, m2. mnIn addition, this learning pattern S is given as a 64-dimensional vertical vector in the case of the consonant pattern.

In the case of a vowel pattern, it is given as a 16-dimensional vertical vector.

Therefore, for the characteristic kernel, each element of a matrix created by multiplying the vertical vector S of the m learning patterns by the horizontal vector S obtained by transposing this vertical vector SII+ is expressed as the m learning patterns. It is calculated by averaging over Therefore, the number of elements of the characteristic kernel is the square of the number of elements of the vector.

Note that in order to obtain the characteristic kernel K that reflects the pattern distribution of the category through such processing, a sufficient amount of learning patterns are required. For this reason, it is necessary to store a predetermined number of learning patterns in the learning pattern memory in advance.

However, in the case of vowels, it is only necessary to prepare learning patterns of 16 dimensions and at least six categories, but in the case of consonants, there is also a 1-inch category, and moreover, it is necessary to obtain 64-dimensional data. For this reason, it is undeniable that a huge amount of memory capacity will be required if the system is left as it is.

Therefore, in order to obtain a characteristic kernel K that reflects the pattern distribution using a small number of learning patterns, the following characteristic kernel update process is performed, and the characteristic kernel is gradually improved to reflect the pattern distribution through sequential calculations. Make it.

That is, the arithmetic processing '+wS S 1 n is repeatedly executed on K=. However, W is a weighting coefficient when updating the characteristic kernel. This weighting coefficient W has a positive or negative value, and if it is positive, it increases the similarity of the characteristic kernel matrix to the input pattern, and if it is negative, it decreases the similarity.

Further, ′ indicates the characteristic kernel before learning the learning pattern S, and K indicates the characteristic kernel updated by learning the learning pattern S.

After that, for the characteristic kernel obtained in this way,
Processing is performed to obtain the eigenvalues and eigenvectors, and based on these eigenvalues and eigenvectors, a standard pattern used in the composite similarity calculation described above is created.

The standard pattern is obtained by performing KL expansion on the above-mentioned characteristic kernel. For example, the standard pattern is obtained by performing KL expansion using power east.

Now, the characteristic kernel K has an eigenvalue λ1. λ2. ~λ□, and the corresponding eigenvector ξ1. ξ2. ~ξ. shall have. In this case, that arbitrary vector U.

is the above eigenvector ξ1. ξ2. ~ξ. A linear combination of U Σ α, ξ.

It is expressed as At this time, Kξ,=λ,ξ.

Since the relationship is established, = ・・・・・・=, Σ α, λ, 8ξ.

becomes.

Here 1λ11〉1λ21〉 ・・・・・・ 〉1λ 1
S u [λ, / λ 1 > 1
Since (i-2, 3, ~, n), when S becomes sufficiently large, the 2 term - 37 - in the above equation converges to O.

Therefore, the above formula Ku=α1　λl　ξ1 It can be regarded as.

s+1 This indicates that the ratio between (K u ) and (K u ) is the eigenvalue λ1. Also (K”
u) is shown to be proportional to the eigenvector ξ1.

By the way, in a calculation process based on such a theory, the results often scale out immediately during the calculation. Therefore, let U be an arbitrary, for example, a unit vector, and v = Ku s + i.

u8+1=(v)/(b)s+1
The following calculation is executed: s+1 (s-0, 1, 2, . . . ). Here (b)o
1 is the element with the largest absolute value of the vector (V), s+
1 Yes. At this time, Ll = (V)/(b)s+1
s+1 so1=(Ku
)/(b)s sol Chi-(Kv)/(b-b)s
Since it becomes sol s,
From this, λl, bs+1. ξ1. It becomes possible to obtain us+1.

Once the eigenvalue λ1 and the eigenvector ξ1, which have the largest absolute value, are determined in this way, the eigenvalue λ2 and the eigenvector ξ2 whose absolute value is the next largest are determined in the same way.

Here, considering K' = one λ1 ξ1 ξ1, from ξ 1 ξ, = O(i-2, 3, ~, n), K' ξ = ξ −λ ξ ξ ξ1=λ ξ
−λ1ξ1 = O (1=1)′ ξ = ξ, −
λ ξ, ξ, 1 ξ.

1 l 1 1 1
l=λ, ξ,
(i≠1). Therefore, ' in the above is 1λ21>...>lλr1>...>lλ. It can be seen that it has an eigenvalue of l>0. It is assumed here that ξ is normalized.

Such processing can be achieved by repeatedly performing the above-mentioned processing on the characteristic kernel converted into -λ ξ·ξ1 (2). Through this processing, eigenvalues with large absolute values and their corresponding eigenvectors are found in order, and dictionary learning is performed.

FIG. 21 shows a dictionary learning process executed based on such a calculation algorithm.

Next, the character recognition section 21 will be explained.

This character recognition unit 21 is composed of a first character recognition block that recognizes characters read by a scanner or the like, and a second character recognition block that recognizes character information input online via a tablet or the like. .

This first character recognition block is an analogy! As shown in FIG. 22, an image memory 21a that stores image data read and input by a scanner,
An area detection unit 21b detects an area in which a character to be recognized is written from the image data stored in the image data stored in the image data tau, and a character to be recognized is detected from the image data tau stored in the image memory 21a according to the area detection result. A character extraction unit 21c that extracts data,' and a standard pattern dictionary 21d.
The recognition unit 21e includes an identification unit 21e that performs character recognition by individually comparing each standard character pattern of recognition target characters registered in advance with the character pattern extracted by the character extraction unit 21c.

This character recognition block is set at a predetermined position on the FAX transmission manuscript paper 21f, for example, as shown in FIG.
It recognizes the characters written in the character box 21g in which the transmission destination is written. The manuscript paper 21f on which such a transmission destination is written is used as the first (first) manuscript when the transmission manuscript consists of a plurality of sheets. Image data obtained by manually reading the first document is stored in the image memory 21a for character recognition processing.

The area detection unit 21b detects the character frame 21g from the predetermined format information of the FAX transmission manuscript paper 21r.
The system obtains position information and detects the area where the character to be recognized is written. The character extraction unit 21c uses this area detection information and the projection pattern information of the image information to individually extract image data of the characters written in the character frame 21g, for example, as shown in FIG. 24. There is.

The identification unit 21e extracts the characteristics of the character pattern from the extracted character image, and compares the extracted character pattern with the features registered in the standard pattern dictionary 21d, as disclosed in, for example, Japanese Patent Publication No. 49-T277a. Pattern matching is performed with the standard pattern of each character. Then, through this pattern matching, the character category of the standard pattern that has been successfully matched is determined as the recognition result.

= 42- It goes without saying that the pattern matching method can be modified in various ways.

By the way, the second character recognition block that recognizes character information input online via a tablet or the like is configured as shown in FIG. 25, for example.

This second character recognition block includes a coordinate detection circuit 21h that sequentially detects a series of position coordinates indicating the writing stroke of a character input online via a tablet or the like.

The time series data of the position coordinates detected by the coordinate detection circuit 21h is input to the preprocessing circuit 21i, and after removing minute noises such as detection errors in the tablet 4,
It is stored in the coordinate series storage circuit 21j and subjected to character recognition processing. In this preprocessing circuit 21i, for example, when one character is input, the size of the character is normalized, etc.

Further, the stroke number detection circuit 21 detects the number of written strokes and the number of missed strokes of the character pattern from, for example, breaks in the written strokes (separations in the time series of position coordinate data).

Accordingly, the recognition processing unit 21m selectively extracts the standard pattern with the corresponding number of strokes from among the standard patterns of the recognition target character category registered in the standard feature pattern memory 2in according to the information on the number of strokes. The characteristics of each stroke of this standard pattern and the coordinate series storage circuit 21j
The stroke characteristics of input character patterns stored in the input character pattern are compared with each other (matching process). Answer determination circuit 2i
p determines the matching processing result, and obtains a recognition target character category having a stroke that corresponds to the stroke characteristics of the input character pattern as the recognition result.

In other words, the input character pattern is recognized by matching the characteristics of the stroke with the stroke characteristics of the standard character pattern according to the characteristics of the writing stroke of the character pattern input online.

As for the characteristics of the stroke, information on position coordinates of end points, intersection points, break points, etc. when a written stroke is approximated by a broken line may be used.

The character recognition unit 21 having the functions described above recognizes character information read and input via a scanner or the like, and character information input online via a position coordinate input device such as a tablet.

Next, the figure recognition section 22 will be explained.

This figure recognition section 22 is configured as shown in FIG. 26, for example. The input unit 22a stores, for example, a captured and input graphic image, and subjects it to graphic recognition processing.

The contour tracing unit 22b divides the tracing direction of the line segment into eight directions, for example, as shown in FIG. There is. Specifically, as shown in FIG. 28, for example, a triangular figure is tracked clockwise, and the information on the tracking direction is stored as, for example, r1, 2, . ~2,3,4. ~4,5,7.
~7'' is obtained as a series of direction codes.

The segmentation unit 22c extracts singular points, such as curved parts, from the sequence of direction codes obtained in this way, and divides the outline of the figure into a plurality of characteristic parts according to the singular points.

The matching section 22d recognizes the input figure by matching the information on the figure outline segmented in this way with the characteristic information of various figures registered in the dictionary memory 22e.

For example, when the figure shown in FIG. 29 is given, from the series of direction codes obtained by contour tracing, for example, three mutually adjacent contour points (1-1) (i)
At (i+1), the sum of the direction codes is calculated in order, and this is smoothed as the direction code at the central contour point i. This smoothing process removes noise components.

Thereafter, the segmentation unit 22c detects end points that are characteristic points of the contour, that is, points where the curve is steep, and divides the contour around the end points.

Then, each divided contour portion is compared with the dictionary memory 22e to obtain the recognition result.

Through the above processing, as shown in Fig. 30, a round figure has no end points, a triangular figure has three end points detected, and a rectangular figure has four end points detected. Each figure is identified and recognized. At this time, information such as that each of the end points is convex, or that the outline connecting the end points is a straight line or a curved line may be used for figure identification.

On the other hand, the image recognition section 23 is configured as follows.

FIG. 31 shows a schematic configuration of this image recognition section 23, which is composed of an original image memory 23a, a binarization device 23b1, a processed image memory 23c, a thinning device 23d, and a code conversion device 23e.

The image memory 23a stores a given recognition target image, and the binarization device 23b binarizes it and stores it in the image memory 23c.

This binarization level is variably set, for example, while monitoring the binarized image on a display.

The line thinning device 23d performs line thinning processing on the binarized image and converts the image into a line figure. The image memory 23c is rewritten with the thinned image and subjected to recognition processing.

The code conversion device 23e is configured as shown in FIG. 32, for example, and first divides the thinned image into a plurality of segments in a segment dividing section 23f. This segment division is performed, for example, by dividing the line figure at its end points, branch points, or intersections. The curvature conversion unit 23g converts the plurality of segments divided in this way,
The curvature of each is found.

Straight line/curve dividing section 23h1 curve dividing section 231. The inflection point dividing section 23j and the inflection point dividing section 23h further divide each segment divided as described above according to the information on the curvature thereof, and by these, the inflection point, the switching point between a straight line and a curved line, Points of inflection, points of radius change in the curve, etc. are detected. Through such segment division and feature point detection, information on each part constituting the image line figure is extracted.

The approximate information creation unit 23m synthesizes information on these divided segments and feature points in the segments to express the image figure, such as the position coordinates of the starting point and ending point of each segment, and the type of the segment. Obtain code information that specifies.

For example, when an input image is given as shown in FIG. 33(a), the image line figure 23n in the input image
is thinned and extracted, and divided into segments as shown in FIG. 3(b). In this example, an image line figure 23n in which a circular figure and a square figure are skewered by straight lines is input. As shown in FIG. 33(b), the image line figure 23n of the lever is divided at its intersection, and segmented into two semicircles, two U-shaped figures, and four straight lines.

The curvature converting section 23g calculates the curvature of each divided segment as shown in FIG. The refraction point dividing section 23j and the inflection point dividing section 23h detect feature points of each segment from the curvature change points. Specifically, in the example shown in FIG. 34(a), the curvature of the two straight lines at the bending point increases sharply, so it is possible to detect the bending point from the change in the curvature. Also, Fig. 34 (b,
In the example shown in ), a change in curvature is detected at the transition portion from a straight line to a curve, so the feature point can be detected from this change in curvature.

Similarly, in the columns shown in FIGS. 34(c) and 34(d), it is possible to detect characteristic points in the segment from the points of change in curvature.

In this manner, the image recognition unit 23 segments the given image figure and detects the feature points of each segment. Then, the image line figure is approximately expressed and recognized as code information indicating each type of a plurality of segments and their position coordinates.

Now, the voice verification section 17 is configured as follows.

This voice matching unit 17 performs individual recognition (
For example, it is configured as shown in FIG. 35.

That is, the speech given through the speech input section 17a is filtered by the phoneme filter 17b and the personal filter 17c, and the speech characteristics are extracted.

Each band of a plurality of channels of the phoneme filter 17b is set by equally dividing the audio frequency band, for example, as shown in FIG. 36(a).

The phonological filter 17b having such filter characteristics extracts feature parameters indicating the phonological characteristics of the input speech. Note that the bandwidth of each channel may be set by dividing the audio frequency band logarithmically.

On the other hand, the bandwidth of each of the plurality of channels of the personal filter 17c is set by exponentially dividing the audio frequency band, as shown in FIG. 36(b). With the personal filter 17c having such filter characteristics, the voice characteristics from the low frequency range to the middle frequency range of the input voice are
This feature is extracted more often than the features on the high frequency side. The filter output of each of these channels is obtained as a feature parameter for personal verification.

Therefore, the word recognition unit 17d uses the phonological filter 17b.
The word indicated by the input speech is recognized from the phoneme feature parameters obtained through the phonological feature parameter with reference to the word dictionary 17e. This word recognition function is performed by the voice recognition unit 1 mentioned above.
9, and the function of the voice recognition unit 19 may be used as is.

Then, according to the word recognition result, a dictionary to be used for personal verification in the personal dictionary 17f is selected. This personal dictionary 17f
is the result of the analysis by the personal filter 17c of a specific word uttered in advance by an individual targeted for speaker verification, classified and registered for each word.

The speaker verification unit 17g calculates the degree of similarity between each feature parameter of the corresponding word selected from the personal dictionary 17f and the feature parameter of the input voice found in the personal dictionary 17c, and calculates the degree of similarity. The values are discriminated using predetermined thresholds. Then, by comparing these discrimination results with each other, for example, a personal category with the highest similarity value and a feature parameter with a sufficient difference from the next highest similarity value is determined to be the speaker of the input voice. Personally identified.

Here, the characteristics of the personal filter 17c will be explained in more detail.As mentioned above, the characteristics are set to be different from those of the phoneme feature filter 17b. Considering the distinctiveness of the individuality of this voice, the distinctiveness can be evaluated, for example, by the F ratio given as F ratio - (inter-individual variance)/(intra-individual fraction).

Now, when considering the F ratio of each channel output of the filter characteristics set in the phonetic filter 17b, an exponential trend is shown by the solid line in FIG. 37.

For this reason, in the past, individual verification has been performed exclusively using voice feature information on the high frequency side.

However, if it is possible to identify individuals using voice features in the entire frequency band, rather than using only high-frequency features of the voice, it is thought that the matching accuracy will be further improved. That is, if the value of the F ratio is 1 or more in all frequency bands and the inter-individual variance exceeds the intra-individual fraction, more accurate individual matching becomes possible.

Therefore, here, as mentioned above, the personal filter 17c
The characteristics of the voice are determined exponentially, and the characteristics of the high frequency range, which is a prominent individual characteristic, are roughly extracted, and the audio characteristics of the low frequency range are extracted finely by increasing the number of channels allocated to the low frequency range. That's what I do.

Specifically, since the change in F ratio of each channel shows an exponential tendency, a filter bank in which the bandwidth of the high frequency channel is increased exponentially compared to the bandwidth of the low frequency channel. and configure this as personal filter 17c
It is said that

According to each channel output of the filter 17c configured in this way, the F ratio is as shown by the broken line in FIG. 37, and a significant improvement in the F ratio in the middle range is recognized. As a result, it becomes possible to carry out individual verification by actively utilizing not only the voice characteristics in the high frequency range but also the voice characteristics in the middle frequency range, and it becomes possible to improve the verification accuracy.

That is, in addition to the features used for word recognition of the input speech, the speech matching section 17 uses a filter bank to extract feature information that clearly reveals the individuality of the input speech. As a result, the individual identification of the speaker, that is, the individual verification, is performed with high precision independently of the phoneme recognition of the input speech.

Next, the speech synthesis section 26 will be explained.

The speech synthesis section 26 includes a discriminator 26a as shown in FIG.
, a decoder 26b, a rule parameter generation device 28c, and a speech synthesizer 2Bd.

The discriminator 28a determines whether the input code string is a character string or not.
Alternatively, it is determined whether it is a code string indicating an analysis parameter for speech synthesis. This information discrimination is performed, for example, by determining the identification information added at the beginning of the input code string. When it is determined that the code string is an analysis parameter, the code string is provided to the decoder 26b, and it is decoded to obtain its phonetic parameters and prosodic parameters.

Further, when it is determined that the character string is a character string, the character string data is provided to the rule synthesis parameter generation device 18c to generate its phonetic parameters and prosody parameters.

The speech synthesizer 26d processes the sound source wave through the vocal tract approximation filter to generate synthesized speech according to the phonological parameters and prosodic parameters thus obtained by the decoder 2[ib or the rule synthesis parameter generation device 26c. generating waves.

To further explain the rule synthesis parameter generation device 2[ic, the device 26c is configured as shown in FIG. 39. The character string analysis unit 26e identifies each word in the input character string with reference to the language dictionary 26, and obtains grammatical information such as accent information, word/clause boundaries, part of speech, and conjugation for the word.

Then, phonological rules and prosodic rules are applied to the analysis results, and control information thereof is generated.

Here, the phonological rules provide information on the pronunciation of the analyzed words, realize phenomena such as rendaku and devoicing that occur due to word concatenation, and generate the phonological symbol strings.

The speech parameter generation unit 28g inputs this phoneme symbol string, and sequentially obtains syllable parameters from the C■ file 2611 according to the syllable unit, and interpolates and combines them. This audio parameter generation section 26g
A phoneme parameter sequence is generated from the phoneme symbol string.

Prosodic rules determine the boundaries of utterances and breath positions according to grammatical information such as word and clause boundaries, and determine the duration of each sound, pause length, etc.

At the same time, this prosodic rule generates a prosodic symbol string that is based on the basic accent of each word and takes into account its clause accent. The prosodic parameter generation unit 2Bi inputs this prosodic symbol string and generates a prosodic parameter string representing a temporal change pattern of pitch.

On the other hand, when the input code string is a code string indicating analysis parameters for speech synthesis, the decoder 2eb functions as follows.

That is, when the code string of the analysis parameter indicates the cepstral coefficient of the CV file, the code string 28m is generally the fourth
As shown in figure 0, the parameter P (pitch) and C8, C1
, ~C, (cepstral coefficients) are assigned bits of -57 and information is compressed. Therefore, the decoder 26b uses a parameter conversion table 26n to convert and decode the information-compressed analysis parameters into a bit number matching the speech synthesizer 28d. For example, each parameter is converted to 8 bits, and the phonological parameter sequence (cepstral coefficients) and its prosodic parameter sequence (
pitch).

The speech synthesizer 2Bd includes a voiced sound source 26q and an unvoiced sound source (M-sequence generator) 26r, as shown in FIG. 41, for example, and generates a voiced sound source wave (P≠0 ), or silent sound source wave (P=O)
are occurring selectively.

This sound source wave is input to the preamplifier 28s, subjected to signal control according to the cepstrum coefficient C of the phoneme parameter, and input to the logarithmic amplitude approximation digital filter 2[it]. This logarithmic amplitude approximation digital filter 26t has a cepstrum coefficient of 01. A resonant circuit is configured to approximate the vocal tract characteristics according to ~Cm, and the above-mentioned sound source wave is filtered. This logarithmic amplitude approximation digital filter 2Eit synthesizes and outputs speech data indicated by the phoneme parameters and prosody parameters.

The signal synthesized by the logarithmic amplitude approximation digital filter 2Gt is then passed through the D/A converter 2[iu, then L
The signal is filtered through P F 2Bv and output as a synthesized audio signal (analog signal).

In the speech synthesis section 26 configured as described above, the speech indicated by the input data series is synthesized according to the rules and output.

Next, the image composition section 27 will be explained.

As shown in FIG. 42, the image synthesis section 27 includes a control computer 27a, a display file memory 27b, an image synthesis circuit 27c, an image memory 27d, and, if necessary, a display 27e.

Note that this display 27e may be the display section 10 that is semi-biased with respect to the workstation.

The image synthesis circuit 27 reads parameters of vectors, polygons, and arcs written in the display file 27b under the control of a dedicated control computer 27a, generates line figures indicated by the parameters, and stores them in the image memory 27.
Writing to the specified address of d. The image generation function of the image synthesis circuit 27 constructs a designated line graphic image on the image memory 27d.

This line graphic image is displayed and monitored on the display 27e under the control of the control computer 27a.

Further, the image generation circuit 27b has a special processing function for image generation and a filling processing function. This special processing function includes, for example, functions such as removing hidden lines from overlapping image figures and performing clipping processing. The filling function consists of filling a partial area of an image figure with a specified color.

Through the functions of the image synthesis circuit 27b, various image figures are created and their synthesis processing is performed.

By the way, as described above, the synthesis of generated image figures and natural pictures can be roughly divided into the following two types. One is the process of embedding and compositing an image obtained by computer processing into a natural picture such as a landscape photograph, and the other is a process of composing a natural picture, such as a landscape photograph, into the background. It consists of the process of embedding and compositing a natural image within an image.

In the case of embedding an image in the former natural image, for example, as shown in FIG. 43, a code indicating "transparent color" is given to the figure generated by the computer. This can be achieved by superimposing and compositing the natural image. Then, in the image area given the "transparent color" code, the information of the natural image will be displayed as is, and in the other parts, the computer-generated figure will be displayed. As a result, image synthesis using a natural image as a background is realized.

This technique is called overlay.

On the other hand, as the concept is shown in FIG. 44, a natural image may be written in the image memory, and a figure generated by a computer may be written on top (in front) of the natural image. This method is called a two-buffer method, and can be implemented relatively easily together with the above-mentioned overlay method.

By the way, in the latter case, in which a natural image is inserted and synthesized within a plane shown as an internal model of a computer, high-speed processing is performed as follows.

The coordinate transformation required to embed a natural image on plane -H into a plane facing in an arbitrary direction in three-dimensional space is given by the following equation.

C4X0C5Y106 However, X and Y are the coordinates on the display surface, and u and v are the coordinates on the natural image.

If this coordinate conversion process were to be executed as is, six multiplications and two divisions would be required each time one pixel is displayed, which would require an enormous amount of calculation and calculation processing time.

Therefore, here, the above-mentioned calculation is divided into two conversion processes and executed using the following intermediate coordinates (s, t). This arithmetic processing is executed at high speed using, for example, affine transformation.

u, = (α S + α2 + αg”) / t
(1) ■ v =, (α S+α + α9)/1s-C5X-
C4Y (2) t-C4X+C5Y
10C6 In other words, perspective transformation is performed using equation (1) above, and then two-dimensional affine transformation is performed using equation (2) to perform perspective transformation to an arbitrary plane at high speed. ing.

Here, since the denominator of equation (1) is the coordinate t itself, it is easy to perform the calculation at high speed by simply slightly improving the conventionally known affine transformation circuit.

- In this way, the image compositing section 27 executes various image compositing processes at high speed.

Next, the output format selection section 24 will be explained.

The output format selection section 24 is activated upon receiving a media selection request signal, and selects which medium is to be used to output data. In other words, it selects which of various media should be used to transmit information.

FIG. 45 is a schematic diagram of the output format selection section 24, in which the media selection control section 24a, the input media determination section 24
b, a partner media determination unit 24C, a media conversion table 24d, and a self-media function table 24e. Also, FIG. 46 shows this output format selection section 24.
This shows the flow of processing. The functions of the output format selection unit 240 will be explained along the flow of this processing procedure.

When the media selection request signal is given, the media selection control section 2.4a requests the control section 2 to provide input media information necessary for media selection operation. Then, it issues a media information detection request and a media function identification request to the input media determining unit 24b.

The input media determining section 24b is composed of a media detecting section 24f and a media identifying section 24g, and detects the manual media given from the control section 2 in response to an information request from the media selection control section 24a, and also determines the function of the detected media. It is designed to identify and judge. For example, when the input media is audio, the input media determining unit 24b identifies and determines that the function of the media is ADPCM.

Thereafter, the media selection control section 24a informs the control section 2 whether the destination of the data output is another functional block of its own terminal (within the workstation) or another function block connected via a communication line or the like. Determine whether it is a workstation or communication terminal.

When an instruction is given to output data to another workstation or communication terminal, the media selection control unit 2
4a requests the control unit 2 for identification information regarding the destination station. In response to this request, information regarding the partner station that outputs data is input to the partner media determining section 24c.

The partner media determining section 24c is a partner station identifying section 24h.

The partner station media identification unit 2411 is configured to include a function identification unit 24j, and operates upon receiving an identification information determination request from the media selection control unit 24a. Then, first, the partner station is identified from the identification information for the partner station, and the media of the partner station is identified. Then, the function of the partner station's media is identified.

Specifically, for example, it is determined that the destination station to which data is to be outputted (transmitted) is an automatic FAX, its communication medium is an image, and its function is G type. Note that this identification of the partner station may be performed based on information sent from the partner station using its negotiation (handshake) function. Furthermore, if there is no negotiation function, it is preferable to provide the function identification section 24j with the media detection function. In this way, it is possible to identify the function according to the media information signal from the other party.

FIG. 47 shows the flow of this partner station identification processing procedure. As shown in this flow, for example, it is determined whether the communication partner station is a telephone or not, and if it is a telephone, it is determined whether or not a FAX signal arrives.

If the other party's station is a telephone and a FAX signal arrives, it is sufficient to identify this as the other party's device being a fax machine. Further, if it is determined that the device is a telephone and no FAX signal arrives, it may be determined that the other party's device is a normal telephone. Furthermore, if it is determined that the device is not a telephone, it may be determined that the other party's device is a communication device other than a telephone.

When the media of the communication partner station is identified in this way, the media selection control unit 24a then refers to the media conversion table 24d configured as shown in FIG. 48, and selects the input media and input function. , obtain media conversion selection information corresponding to the destination device, the destination device media, and the function of the destination device.

For example, the input media is audio and its function is AD-PCM.
If the destination device is a Gm type fax, the media of the destination device is an image, and the main media conversion function is (voice) to (code character).
・= 67- (code character) to (image) etc. is required. At the same time, it is required that the conversion function can be realized by (ADPCM, voice) to (GIII; FAX). At this time, if dependent media conversion information exists, this information is also obtained at the same time.

Whether the media conversion information obtained in this way is
and selectively specifying the format of the data output.

In addition, when data output is to be performed inside the own workstation, the media selection control unit 24a refers to the own media function table 24e to find an output format in which the data can be output. According to this information, the media selection control section 24a refers to its own media conversion table of the media conversion table 24d, similarly obtains media conversion information, and provides this to the control section 2.

According to the media conversion information obtained in this way,
For example, the above-mentioned speech synthesis section 26 may be used to convert text information given as a character code series to speech information and output as data, or the speech recognition section 19 may be used to convert speech information to character code series information. The data will be output.

Next, the database section 32 will be explained.

The database unit 32 organizes and stores various data such as codes, images, and sounds, and provides this data to various application systems. Figure 49 shows this database section 3.
2 shows a schematic configuration of an interface section 32a that executes command analysis processing, etc., a data operation section 32b that executes database search processing, etc., 2 a magnetic disk device 32c or an optical disk device as a storage medium for storing various data. 32d, and its additional functional section 32e.

Various data are classified and organized into a plurality of relations according to the data type, and a database is constructed by registering each relation.

The database unit 32 will be explained below by dividing it into four parts: its logical structure, stored data, physical structure, and additional functions.

The logical structure indicates how various data are stored when the database unit 32 is viewed from the application system side. Here, data is handled as a logical structure according to the relational model, for example, as an image of a table as shown in FIG.

A table (relation) has several columns (attributes)
Data in a predetermined unit is stored in each column. The unit of data (Tarapur) is
It is defined as one set of values to be stored in each column. One relation is constructed by an arbitrary number of attributes storing such Tara pulls.

However, in this model, data is stored in the database by specifying a relation name and giving values for each of its attributes. Database searches also specify relations and attributes, and determine whether the value stored there satisfies a predetermined condition with the specified value or the value stored in another attribute. This is done by extracting Tarapuls that satisfy the conditions.

This search condition is given as the values being equal, unequal, smaller, larger, etc. At this time, it is also possible to specify search conditions for each of a plurality of attributes and perform logical processing (AND, OR, etc.) on the results of the condition determination. Furthermore, it is also possible to perform a database search to find a predetermined combination among multiple relations by specifying multiple relations and using conditions such as the value of the attribute of one relation being equal to the value of the attribute of another relation. be.

Furthermore, data deletion from the database is basically performed in the same manner as the above search, but instead of extracting the data, the data is deleted by deleting the data.

Furthermore, the same applies to data updating, which is performed by changing the value of a specified attribute of the obtained Tara pull and storing it.

In addition, in each relation, information (person name, person in charge code), etc. of the person who is permitted to read, add, or change data for each attribute is entered, and data protection measures are taken. Note that instead of taking this data protection measure for each attribute, it is also possible to take it for each relation.

It should be noted that there may be more than one person's information listed here.

In the relation example shown in FIG. 50, the data is shown as a character string, but the data stored in each relation may also be a simple bit string. In other words, the data stored in the relation may be not only character strings, but also image information, audio information, and the like.

Now, the data stored in this database includes, for example, the "one-person schedule" shown in Figure 50 mentioned above, as well as the "address book" shown in Figure 51.
“Individual work and its agents” “Operation history” “Human resources”
It consists of various "relationships" such as "conference room,""conference room reservation," and "conference."

As shown in this example, rations consist of rations that are mainly used for personal use and rations that are commonly used by many users. A personal partition is provided for each workstation used by each individual, and a common partition is provided for a workstation that is common to a plurality of users.
・ .

Note that a common workstation does not necessarily mean that its hardware is different from other workstations. It goes without saying that a personal workstation may also serve as a common workstation. Furthermore, the number of common work stations is not limited to one, but may be multiple depending on the hierarchical level of the system. In short, it is possible to easily identify a common work station from among multiple work stations. A common workstation is configured as possible.

Here, the data structure of the "personal schedule" relation shown in FIG. 50 will be briefly explained. −・ From this relation, the relation name.

is the "personal schedule" and was created by "△△, △△". It is shown that , this relay creator "△△△△" is permitted to perform all data operations on this relay region.

Also, according to the data protection function added to this relay, all 11J members are permitted to read the data, and data additions are allowed by ``OO○○'' and ``I belong to the engineering department''. Permission is granted only to those who do. In addition, this "Runi who belongs to the technical department" is, for example, "Human Resources."

It can be found by referring to the ration. Additionally, 9 data changes are permitted for those whose "person level" value is "5" or higher. "This one-person level" refers to personnel relations, for example (general manager;
) (Deputy Manager; 7) (Section Manager; 6) (Chief: 5), etc.

Furthermore, this relation has "start time" and "end time."

End time" f Type J Attributes such as "name" and "location" are set, and data is written to each of them.

Next, the physical structure for actually storing the above-mentioned various glue ratios in this database section 32 will be explained.

The information storage unit (storage unit) stores a large amount of data and can read and write any part of it at relatively high speed, and is not expensive at all, such as the magnetic disk device 32c or optical disk drive described above. Apparatus 32g is used.

・In order to store the database in this information storage unit, the storage area of the information storage unit is divided into sections of a specific size (for example, several kilobytes) (determined according to the length of the data pull, the speed of the computer, etc.), and each section is divided into pages. It is managed as follows.

As shown in FIG. 52, for example, page management information is stored in the 0th page, information on the relation list is stored in the 1st page, and information on pages in use is stored in the 2nd page. ' This list of relations indicates the location of various relations in the database text.

For example, the actual data stored on the 9th page and the 11th page is stored on the 5th page, and is sorted based on the ration attribute (main attribute) and according to the information on the index page stored on the 10th page. It has become. This index page information indicates in which page the number of attribute values is stored.

When searching data using an attribute other than this main attribute, a second
Via the sub-index of page 0, first the 21st
Get the sub-data shown on the page and the 2nd z page. This sub-data contains only the attribute values and the above-mentioned main attribute values, and the actual data is determined using the attribute values determined here.

In addition, if the amount of actual data is enormous, such as image data or audio data, and some bit errors in the data are not a problem, these 5 actual data may be stored on another device such as the optical disk device 32d. The information may be stored in an inexpensive information storage device. In this case, the 9th page or the 1st page
The actual data page, such as page 1, may store information to that effect and the storage location of the actual data in the device.

However, additional functions for the database constructed in this manner include, for example, automatic disposal of unnecessary data.

This automatic discarding of unnecessary data is performed by providing [discarding; allowed/disabled] and [discarding method] as additional information of the relation, and by operating an erase command for each relation at predetermined intervals.

Incidentally, erasing of the table can be performed by determining, for example, whether or not the end time of conference information is before the current time. Therefore, there is no need to add any special functionality to erase such a Tara pull.

Another important additional function is data security. This data preservation function prevents data from becoming fraudulent (random or lost) due to, for example, hardware failure or power outage. Specifically, this data preservation function is realized by duplicating information, writing magnetic data, and the like.
'In this way, the database section 32 classifies and organizes various types of data by relation, and manages the data on a page-by-page basis to provide it to various application systems.

- Next, the work environment data collection section 25 will be explained.

The work environment data collection unit 25 collects data on past operation history for the workstation, and provides operation guidance based on this data.゛Here, the work environment data collection unit 25 includes commands corresponding to the functions of the information processing system, as shown in FIG. 53, for example, and
A command correspondence table is provided that associates commands corresponding to functions of other information systems.

Specifically, when the information processing system is A1 and the other information processing systems are B, C, D, etc., the command of the other system corresponding to the command "DELETE" in system A is "DEL", etc. ERASE""RE
"MOVE" is indicated by the command correspondence table.

Figure 54 was input by the user; Analyzing the cloak 2
, which shows a schematic configuration of a work environment data collection unit 25 that executes predetermined operations and various guidances.

In this work environment data collection section 25, first, the command inputted from the command input section 25a is sent to the command analysis section 25.
b and is analyzed with reference to the command correspondence table 25c. Specifically, it is checked whether the input command is registered in the command correspondence table 25c according to the procedure flow shown in FIG. That is, when a command is input, it is first checked whether the human-powered cloak belongs to system A or not. And the input command is system A
When the command is analyzed as a command, the command analysis section 2.5b gives the input 1 command to the command execution section 25d, and causes the command execution section 25d to execute a predetermined operation based on the command.

On the other hand, if the input command is not from system A, it is checked whether it corresponds to a command from another system, and if there is an associated command,
The corresponding command is displayed on the screen display section 25e. In other words, if the command used in another system (system B) is, for example, "DEL", the corresponding system A command "DEL
” and uses this as operation guidance on the screen display section 25.
e It will produce ascus q.

Note that if the command corresponding to the input command does not exist in the command correspondence table 25c, the screen display section 2
A command error message is displayed in step 5e.

Specifically, the command input is processed as follows. Now, it is assumed that a user who has experience operating systems B and C operates Shouginba system A (the information processing system). If you enter the Utilization Labor Saving 5 command here and delete the data "ABC", the conventional 6Maki system 4
"D E" for data deletion according to the instruction manual of A.
It is necessary to search for the commands "L, E T E" and enter them all.

However, in this case, the user has created a data deletion command based on past experience, for example, the data deletion command used in System C.
“E RA S, E to BC” in Figure 56 (a) (
Enter as shown.

Then, the work environment data collection unit 25 analyzes this input code, identifies the command "DELETE" of the system A that corresponds to the input command "E-RA CE" in the command correspondence table 25c, and converts it into a code. It will be displayed as As a result, even if the user is operating Stem A for the first time, the user knows that the data deletion command is "I) F, LETE", and by inputting that command according to the guide, the user can It is now possible to erase data.

Also, in order to display a list of file names, for example, the 56th
As shown in Figure (b), when the command "DjR" is input in system B, the corresponding command "CA" in system A is input in the same way! 'A
” is requested and displayed as a guide. As a result, by inputting the command “CATA” according to this guide, 1. A list of the file names will be displayed.

In the second step, by utilizing the function of the environmental data collection unit 25, by inputting commands and commands used in a certain system in the past, the corresponding commands in that system can be displayed as a guide. And φ. Therefore, the system user can operate the system by making maximum use of the knowledge acquired in the past. Then, it becomes possible to easily know the commands of the information processing system. Therefore, the user is freed from the trouble of checking the operating manual of the information processing system each time. Therefore, effects such as being able to significantly shorten the time required to learn how to operate the system can be expected.

Incidentally, when a command corresponding to an input command is obtained and the command is not displayed as a guide, the command may be executed after receiving a judgment input of its pass/fail.

That is, the procedure flow is shown in FIG. 57, and the display example is shown in FIG. 58.
ERASE", and when the corresponding deletion command "DELETE" of system A is requested, it is checked whether this is correct or not.

When a positive (Y) instruction is input, it is determined that the input command indicates "DELETE", and this is sent to the command execution unit 25d to execute the process.

In this way, the command correspondence is guided and at the same time the desired process is executed according to the input command, so there is no need to input the correct command again. In other words, the input command is automatically converted into a corresponding command, and the process is executed. Therefore, it becomes possible to further improve the operability.

Note that any number of compatible commands may exist depending on the type of system. In short, the command pairs 83 are stored in association with each other in the table 25c.

It's fine if you do. It goes without saying that the command is not limited to the above-mentioned character string format.

Next, the collection of system proficiency data by the work environment data collection unit 25 will be explained.

FIG. 59 is a flowchart showing the system proficiency data collection process.

When a user enters their identification code (user number, password, etc.)
When input, the work environment data collection unit 25 obtains the proficiency level corresponding to the identification code from the external storage device and sets it inside the device. This proficiency table stores the degree of proficiency of each user with respect to various functions of the system, and is structured as shown in FIG. 60, for example. In other words, this i-seasoned skin table records for each function used the frequency of use of cedar, the date and time of last use, the proficiency class for the function declared by the user, the proficiency class when the function was last used, and It is composed of information such as the complexity of the function.

Here, the degree of complexity increases as the function to be used requires specialized knowledge, and as the function becomes more advanced than the basic function. “However, such a learning skin table is set up for each user,
``They are each stored in an external storage device.

Incidentally, for a user who uses the system for the first time, a learning curve for that user is created by setting a new identification code and is registered in the external storage device.

In the external storage device, for example, as shown in FIG. 61, in addition to the above-mentioned threat skin table, messages for each function used corresponding to the proficiency class are registered. The lower the proficiency level of the message, the easier it is to understand the message, including the background explanation. The content of the class is advanced, including simple explanations and introductions to specialized functions.

Also, proficiency classes are, e.g. A; Beginner class B. Intermediate class C7 expert class The classification is set as follows.

When the proficiency level corresponding to the input identification code is obtained, a menu is displayed to allow the user to select the function to be used. In this menu, the user inputs, for example, a number corresponding to the function to be used. Then, the control section determines whether the input information is an end signal or a selection signal for a function to be used, and if it is a selection signal for a function to be used, the operation is as follows.

When a function selection signal is input, the user first refers to the proficiency table for that user and determines the frequency of use, date and time of last use, declared proficiency class, etc. corresponding to the selected function. information is frozen.

Then, weighting is performed according to this information, and the current proficiency class is determined.

To determine this proficiency class, for example, the usage frequency is Pl, the minimum usage date and time is T, and the current usage date and time is 1.

e Date and time are T 1 User declared proficiency class is Xl, front usage proficiency class is X2ζ (A, B, C1, complexity is P
1, and when the discriminant function is F, it is determined as r F =, P, + 2 (To-To) r 10 3G1 [X1] 10 4G2 [X2] + 5Pc. However, in the above equation, KKl, 2°K 3 , and K 4 are constants that are set to appropriate values through experiments or the like. Further, the above GG is 1.2, Yl, Y2. Y3. Zl, Z2. Z3 is A,
B.

This is the evaluation weight for C. These evaluation weights are Y, Z2, Z31. 2.
3, and is set to an appropriate value through experiments, etc.

Here, G1 [X, ] means that it takes the value Yl when X1=A, and takes the value Y2 when X2=B. Further, (To-To) is the number of days from the date and time of last use to the present time converted into time.

Therefore, class determination is performed using the above-mentioned discriminant function F.

This is done as follows depending on the value of .

F <N...A class N ≦F <N...B class 1, r
2 N2≦Fr . . . Class C Note that the determination thresholds N, , N2 are appropriately determined based on experiments and the like.

Once the proficiency class is determined in this manner, guide messages and error messages that correspond to the determined proficiency class and correspond to the specified usage function as described above are obtained from the external storage device.

Thereafter, the currently determined proficiency class is compared with the previous proficiency class stored in the proficiency table. If there is a change in the proficiency class, a message indicating that the proficiency level has changed is added to the guide message and written.

This proficiency class change message consists of four types of messages as shown in FIG. 62, for example. [7] The information is determined according to the form of the class change, and is displayed together with the guide message and the like.

The user performs processing operations according to the various messages displayed in this way.

Specifically, when it is determined that the user is in the beginner class (class A) for the function used to store created data in a file, a message as shown in FIG. 63 is displayed.

If the user makes a mistake in inputting information despite this message, an error message as shown in FIG. 64 is displayed, for example, and instructions for operating the function to be used are provided.

Further, if the user's proficiency level is determined to be intermediate class (B class), a message as shown in FIG. 65 is displayed. If the user makes a mistake in inputting information in spite of this message, an error message as shown in FIG. 66 is displayed, for example, and guidance on how to use the function is provided. Similarly, if the user's proficiency level is determined to be in the expert class (C class),
A message as shown in FIG. 67 is displayed, and if there is an error in the information input, an error message as shown in FIG. 68 is displayed to guide the operation of the function to be used.

When data is input into the blank field of the guide message displayed as described above, the control unit calculates the frequency of use of the proficiency table of the user as described above.
+1) At the same time, the date and time of last use and the previous use proficiency class are updated. Then, it prompts the execution of the function to be used, assumes that the function to be used has been completed, and returns to the menu display operation for selecting the function to be used.

If the usage function selection signal is input again here, the above-described process will be repeated again.

However, when an end selection signal is input, the proficiency table created and updated in duplicate is written to the proficiency file in the external storage device along with the identification code of the corresponding user.
Save this. Then, the series of processing procedures is completed.

In this way, the work environment data collection unit 25 collects data on proficiency regarding system operations and provides appropriate guidance for the operations based on the collected data.

The above is the basic configuration of this workstation and its functions.

Next, the video creation function of this workstation will be explained.

The function of creating a video based on this text data is performed by the control unit 2.
This is realized by using the processing functions of each section described above under the control of. For example, with respect to documents used in offices, etc., the character strings of text data are analyzed and videos are created to explain the content shown in the text data in an easy-to-understand manner.

FIG. 69 shows the procedure flow of moving image creation processing. Which processing procedure is started by inputting the text ID of a specific text to be processed by +1 (step a). Then, the control unit 2 uses the specified text ID.
Accordingly, the corresponding text data stored in the database section 32 is read out (step b), and a character string representing this text data is displayed on the display device (display section) 1'0 (step c).

For the text data (character strings) displayed in this way, a text section related to video creation is specified. This text section is specified by controlling the cursor or using a light pen. Since a moving image with a temporal sequence is usually created, a plurality of sentence sections are each specified for the text data displayed on the display (step d).

Note that this text section specification involves specifying a text section range that specifies the start and end points of the text section, and specifying a link for a text section that indicates the connection loss of multiple text sections that are closely related to each other. Consisting of

Specifically, for example, a text section for text data is specified as follows.

7 Yamada Kogyo Co., Ltd. had sales of 25 billion yen in fiscal 1975. After that, business performance improved.
7 In FY 1980, the amount increased by 8 billion yen from FY 1975.B
, MaCta.

However, in 1985, due to the impact of a decline in demand, the sales in 1985 decreased by 15-92 compared to 7 in 1988.
= D % decreased. The country is currently experiencing the effects of economic fluctuations. ” As shown in this example, the start point of a sentence section is specified by M, and the end point is specified by M. Then, for each specified sentence section, the management data A, B, C,
D, . . . are attached. The link is specified by, for example, associating specific characters of each text section with such text sections.

However, once the text sections are identified in this way, the character strings are analyzed for each text section, and text information for creating a video is extracted (step e). Of course, the analysis of the character string in this sentence section is performed by taking into account the link information described above.

Extraction of data for creating a video by analyzing text data is performed, for example, by dividing the character string in the text section into word units (step el) and analyzing the syntax (step e2), as illustrated in FIG. ). Through this syntactic analysis, the subject part, predicate part, modifier part, etc. corresponding to each character string in the sentence section are determined.

Thereafter, the subject of the sentence is identified based on the syntactic analysis result (step e3), and the subject is further identified (step e4). Furthermore, the temporal relationships of various numerical data contained in the text data are analyzed (step e5), and the meaning of the predicates thereof is analyzed (step eB).

The subject and subject matter of the above processing procedure.

Numerical data included in the sentence section is numerically processed according to the information on the temporal relationship and the meaning of the predicate (step e7).

This numerical processing extracts the data necessary to create the video.

Specifically, in the case of the text data mentioned above, the text section A
``Yamada Kogyo Co., Ltd.'' is identified as the subject, and ``Sales High'' is identified as the subject. In addition, in sentence sections B, C, and D where the subject or subject cannot be identified,
The analysis process proceeds assuming that the subject and subject are the same as the previous sentence section.

Therefore, time analysis extracts the part that expresses the time contained in the modified part and obtains information about that time.

For example, from the character strings ``in 1985'' and ``compared to 1985,'' the information at that time is ``mainly in 1985,'' or ``5 years before that was proposed as a comparison.'' Extracted as "

In the predicate semantic analysis process, words such as "increased" and "decreased" are extracted as the semantic analysis results.

The numerical processing creates, for example, the following video creation data from the information obtained in this way.

(a) Designation number of video model; 1 (b) Subject; Sales (c) Numerical value of subject, 250. '33
0. 280(d) Subject unit: Billion yen (e) hourly figure, 50.55.60 (f
) Unit of time; year (g) Subject: Yamada Kogyo Co., Ltd. The designated number of the video model above indicates which of the multiple models prepared as video expression formats will be used to create the video data. It is to be specified. Specifically, model number 1 designates a product model, and model number 2 designates a shape change model.

After the moving image creation data is extracted in this way, moving image data is created according to these data (step f). Then, the created moving image data is displayed on the display (step g), and the moving image data is stored in the database.

Here, the creation of video data will be explained using a product model as an example. The video creation section is configured to include an image generation section 41, a video model database 42, a video storage control section 43, and an image database 44, as shown in FIG. 71, for example.

The video model database 42 stores, as product models, computer graphics images, multi-valued images input from the TV left camera, and graphical images thereof as graphical images, as well as control information for controlling the graphical images. is stored.

The image generation unit 41 inputs the moving image creation data as described above, and learns the form of the moving image to be created from the moving image model number. If a product model is specified, a model corresponding to the video generation environment is selected from among the plurality of models prepared in the video model database 42.

The image generation unit 41 first sets a scale for the display screen according to information such as the unit of the subject and the unit of time. Then, on this scaled display screen, a desired figure is generated based on the numerical value of the subject, the numerical value of the time, etc. mentioned above.

At this time, the graphic display format is not limited to one type, and a method appropriate to the situation is adopted as appropriate.

Specifically, the trend in sales is represented by a graph of bars, broken lines, etc., and changes in the numerical data are represented by other states, such as by changing the display color.

Through such image generation processing, a moving image corresponding to the content of each sentence section described above, for example, the seventh
As shown in FIG. 2, numerical data expressed in a frame shape is formed as image data that changes in sequence. In other words, a graphic image of the product model or moving image data whose attributes are variably controlled is created according to the information on the temporal history of the text section.

Specifically, in the above-mentioned example of text data, the moving image of text section B is generated as moving image data in the process of sequentially increasing sales compared to 1975. In this case, since sales are increasing, the graphic information is displayed in blue. Also linked text section C
, D, the sales amount is decreasing, so a moving image of the process of the sales amount decreasing sequentially is created, and its graphic information is displayed in red.

That is, from the aforementioned text data, a moving image is created that shows changes in numerical data over time for each text section.

By the way, in the conventional system, sales data for each year was simply displayed as a graph as a still image. Even if a plurality of still images corresponding to each sentence section were sequentially displayed, changes in sales for each year were only expressed in steps.

In comparison, according to the video image mentioned above, how the numerical data changes for each sentence section is expressed as a video image, so the process of change is very intuitive. It becomes easier to understand.

The video storage control unit 43 stores the video data created in this manner in the image database 44 in association with the text data described above, thereby making it possible to reuse the video data.

As explained above, the video creation function of this workstation not only extracts the numerical data of a specified text section and obtains its graphical information, but also extracts information such as the temporal relationship of the text section. It extracts the numerical data, analyzes the changes under which the above numerical data was obtained, and expresses it as a moving image. Therefore, it is possible to understand very easily how the numerical data expressed by figures, etc. was obtained, for example, in terms of time, by means of the moving image representation. Incidentally, there is a problem with expressions using still images, such as the fact that the numerical meaning cannot be understood unless the image is compared with another still image. Therefore, unlike conventional systems that simply create still images for each sentence section and display them sequentially, it is possible to express the meaning of each sentence section very effectively and understand the content of the sentence data. It has great practical effects, such as making it easier.

Note that the present invention is not limited to the embodiments described above. For example, as a representation format for a moving image, not only the shape of the figure may be changed, but also the display color and display brightness may be changed. Furthermore, there is no limitation as to what kind of figure can be used as the product model. Furthermore, a moving image may be generated by changing the volume or shape of an object using a shape change model. Furthermore, it is of course possible to express the temporal process by making the process of change rapid or slow. In addition, it is possible to create video data using various video expressions.

Furthermore, the algorithm for analyzing character strings in a designated text section, the process for extracting data for creating a moving image based on the analysis results, and the processing algorithm for numerical data can be changed in various ways according to the specifications. In short, the present invention can be implemented with various modifications without departing from the gist thereof.

[Effects of the Invention] As explained above, according to the present invention, based on the information of the specified sentence section of the sentence data, the video data expressing the content is changed, for example, to the numerical data accompanying the temporal history. Created while expressing change. Therefore, compared to providing information using still images, for example, the process of change in numerical data is expressed in a much easier to understand manner, making it possible to effectively assist in understanding the content of text data. Great practical effects can be achieved.

[Brief explanation of drawings]

The figures show one embodiment of the present invention. Figure 1 is a diagram showing characteristic processing functions of a workstation according to the present invention, Figure 2 is a schematic configuration diagram of the workstation, and Figure 3 is a diagram showing the workstation. Figure 4 is an exploded perspective view showing the structure of the IC card, Figure 5 is a diagram showing the structure of the printed circuit board of the IC card, and Figure 6 is the semiconductor integrated circuit part of the IC card. 7 is a diagram showing the configuration of the encryption processing section in the workstation, FIG. 8 is a diagram showing the concept of encryption/decryption, FIG. 9 is a configuration diagram of the encryption section, and FIG. Figure 11 is a diagram showing the configuration of the decoding unit, Figure 11 is a diagram showing the configuration of the R3A processing unit, Figure 12 is a diagram showing the configuration of the image matching unit in the workstation, and Figure 13 is a diagram showing an example of a face subjected to image processing. , Fig. 14 is a diagram showing the structure of image data, Fig. 15 is a diagram showing the configuration of the speech recognition section in the workstation, Fig. 16 is a diagram showing an example of an input speech pattern, and Fig. 17 is a diagram showing the structure of the consonant. 18 is a diagram showing an example of a transition network; FIG. 19 is a diagram showing a procedure for speech recognition processing; FIG. 20 is a diagram for explaining partial interval detection for input speech; Figure 21 is a diagram showing the learning processing procedure of the speech recognition dictionary, Figure 22 is a diagram showing an example of the configuration of the first character recognition block of the character recognition unit in the workstation, and Figure 23 is the description of the character strength j to be recognized. 24 is a diagram illustrating the process of cutting out characters to be recognized. FIG. 25 is a diagram illustrating the configuration of the second character recognition block in the character recognition unit. 26 is a diagram showing the configuration of the figure recognition section in the workstation, FIGS. 27 to 30 are diagrams for explaining the figure recognition process, FIG. 31 is a diagram showing the configuration of the image recognition section in the workstation, Figure 32 is a configuration diagram of the code conversion device,
Figure 33 is a diagram showing an example of processing for an input image;
Figure 4 is a diagram showing feature point detection in a segment, Figure 35 is a diagram showing the configuration of a voice verification unit in a workstation, Figure 36 is a diagram showing an example of band division of a filter bank, and Figure 37 is a diagram showing filter characteristics. Fig. 38 (From Fig. 38--Sound platform in Kusteshin,
Figure 39 is a diagram showing the configuration of the rule synthesis parameter generation device, Figure 40 is a diagram showing the speech parameter conversion structure, Figure 41 is a diagram showing the configuration of the speech synthesizer, and Figure 42 is the workpiece. 43 and 44 are diagrams illustrating the concept of image compositing processing. 45 is a diagram illustrating the configuration of an output format selection unit in the workstation.
47 is a diagram showing the flow of the output format selection process, FIG. 47 is a diagram showing the flow of the partner station identification process, and FIG.
Figure 49 shows the structure of the database part in the workstation, Figure 50 shows the data structure of the database, Figure 51 shows the structure of the meso-ear conversion table. Figure 5 showing an example
Figure 2 shows the structure of the ration, Figure 53 shows the structure of the command correspondence table, and Figure 5 shows the structure of the command correspondence table.
Figure 4 is a diagram showing the configuration of the work environment data collection unit in the workstation, Figures 55 to 58 are diagrams for explaining the processing of the command unit, and Figure 59 is a diagram showing the flow of the system proficiency data collection process. Figure 60 is a diagram showing the structure of the proficiency level table, Figures 61 to 68 are diagrams for explaining the processing of the work environment data collection unit, and Figure 69 is a diagram showing the moving image generation processing in this workstation. A diagram showing the flow, FIG. 70 is a diagram showing a procedure for extracting data for creating a moving image by analyzing a specified sentence section, FIG. 71 is a diagram showing an example of the configuration of the moving image creating section, and FIG.
The figure is a diagram showing an example of a moving image generated corresponding to a sentence section. DESCRIPTION OF SYMBOLS 1... Bus, 2... Control unit, 3... Image input device, 4... Position input device, 5... Audio input unit, 6...
- Keyboard 1 part, 7... IC card part, 8... Bus controller, 9... Audio output device, 10... Display part, 11... Image output device, 12.13
...Communication device, 14.Switching device, 15.Timer part, 16.Encryption processing section, 17.Voice verification section,
18... Image matching unit, 19... Voice recognition unit, 2
0...Speech analysis section, 21...Character recognition section, 22...
・Graphic recognition unit, 23... Image recognition unit, 24...
Output format selection unit, 25... Working environment data collection unit, 2
6... Voice synthesis section, 27... Image synthesis section, 28
... Graphic synthesis section, 29 ... Audio compression/expansion section, 3
0... Image compression/expansion unit, 31... Signal processing unit, 32... Database unit, 41... Image generation unit,
42...Movie model database, 43...Movie storage control unit, 44...Image database. Applicant's Representative Patent Attorney Takehiko Suzue No. 1 Case 11 Japan HG3-27G672 (30) Figure 2 ( ) Figure 21 Figure 24 Figure 53 Figure 54 Figure 55 Figure 65 Figure 66 Figure 67 Figure 68 Figure 69 (Data for middle point PL) Figure 70

Claims (4)

[Claims] (1) A means for displaying text data as a character string on a display device, a means for indicating a specific text section in the displayed text data, and a means for analyzing the character string of the indicated text section and An intelligent workstation characterized by comprising means for extracting text information, means for generating and displaying a moving image according to the extracted text information, and means for storing information on the moving image. (2) The intelligent workstation according to claim 1, wherein the text section is specified by specifying a plurality of text sections and their link relationships. (3) Extraction of text information is performed by parsing a character string in a specified text section and extracting information on the main body, subject matter, numerical data, and temporal relationship of the text. Intelligent workstation according to paragraph 1. (4) The intelligent workstation according to claim 1, wherein the moving image is generated by changing the information of the displayed graphics according to changes in numerical data and their temporal relationships.