JPH06175893A - Character code converting device and document retrieving device using the same - Google Patents

Abstract

PURPOSE:To provide the character code converting device and character string retrieving device for performing collation processing at high speed even concerning a text which one-byte characters and two-byte characters are mixed. CONSTITUTION:This device is provided with plural character code fetching means 410a and 410b (windows A and B) for fetching the bytes of characters two by two in the state of shifting them one by one, character code selecting means 420 for selecting a character code to be outputted based on the preceding selected result and the code converted result from among character codes, and code converting means 430 for discriminating a character type showing whether the character code is the one-byte character or the two-byte character and for converting this character type to the bit compressed character code system.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a full-text search in an information processing system, particularly an information search system, and a high-speed character string collation even for a text in which a 1-byte character code and a 2-byte character code are mixed. The present invention relates to a character code conversion device and a document search device that realize the above.

[0002]

2. Description of the Related Art In the field of information processing systems, a certain character string (hereinafter, referred to as a search term) designated by a searcher is included from a group of documents (hereinafter, referred to as text) including character string data. Finding all the documents is one important process.

Several document search devices have been proposed to realize such a search system. The structure of a typical document retrieval device among them is shown in FIG. 2, and the contents will be described.

(LAA Horror: "Hardware Systems For Text Information Retrieval", ACM, SIG, IR, 6th Conference 1983
Year, L. A. Hollaar: “Hardware
systems for Text Informat
Ion Retrieval ”, ACM SIGIR6
(th Conference 1983) In the document search device 1, the search control means 101 controls the search device as a whole and communicates with the host computer. That is, the search request 201 sent from the host computer is accepted, analyzed, and sent as search information 202 to the character string collating means 102 and the composite condition judging means 103. Also,
The search control means 101 controls the storage device control means 104 to control the text 204 stored in the character string storage means 105.
Is read out to the character string collating means 102.

The character string collating means 102 uses the text 204
Information 2 that checks whether or not there is a character string that matches the search request 201, that is, a search term, and if there is a corresponding one, identifies the character string that corresponds to that search term 2
05 is output to the composite condition determination means 103. The compound condition judging means 103, for the search term identification information 205,
It is checked whether or not the logical condition composed of AND and OR specified in the search request 201 is satisfied. When the specified composite condition is satisfied, the identification information of the corresponding document is returned as the search result 206 to the host computer.

As a character string collating method in the character string collating means 102, which is the key to the document retrieval apparatus 1 described above, a method of searching a plurality of retrieval terms by one text scanning using a finite automaton is known. There is. A typical method is the method proposed by Aho et al. (A.V.A.H.O. and M.J.Collasic: "Efficient String Matching", Communications ACM, Vol. 18, Vol. 6
No., 1975, A.S. V. Aho and M. J. Co
rasick: "Efficient String"
Matching ", CACM, VOL. 18, No.
6, 1975), and as hardware for performing high-speed collation processing on an input text consisting of a character represented by a 2-byte code (hereinafter referred to as a 2-byte character) by using the automaton by Aho. -1050
No. 40 publication is proposed. The character string collating circuit described here will be described with reference to the block diagram of FIG.

This prior art is based on the character code registers 211 and 213, the switching circuit 212, and the state transition table 22.
0, state number registers 214 and 250, and a discrimination circuit 260.

The character string collating operation of this prior art will be outlined below.

First, as an initial setting, the state transition table 2
The automaton for matching the search term specified in 20 is stored. Further, 0, which is the initial state of this automaton, is set in the state number register 214.

The matching operation starts by fetching double-byte characters from the text 204 character by character into the character code register 211. In the character code register 211, the captured character code is the upper byte 300 and the lower byte 3
The data is divided into 01 and is alternately set to the character code register 213 via the data line 302 by 1 byte alternately in the order of high order and low order. The state transition table 220 is the output 3 from the character code register 213.
03 and the output 30 from the state number register 214 that stores the state number in the current state (hereinafter referred to as the current state number)
7 is accessed as an address. This character code 3
03 and the current state number 307, each entry of the state transition table 220 indicates the state number of the transition destination, a specific value indicating that the search term is matched, and a compulsory transition to the initial state 0. One of the specific values to be stored is stored and is output as the state number 304. The state number 304 is a new state number 305 (hereinafter referred to as the next state number), and the state number register 250
Held in. In the discrimination circuit 260, the status number register 2
It is determined whether or not the search term is collated by looking at the status number 305 output from 50. Ie state number 305
When is a specific value indicating the collation of the search term, the collation result 205 is output. If it is not a specific value, the state number 305 is stored in the state number register 214 as a new current state number 306. If this value represents a forced reset to the initial state 0, the new current state number 3
0 is stored in the state number register 214 as 06.

A character string collating operation is realized by repeating the above series of operations.

The operation of this prior art will be described below with reference to a specific example. The automaton shown in FIG. 4 is used as an example.

This figure shows an automaton for collating (0x3324) (0x4834) in the code representation using "sea level", a hexadecimal number given as a search term from the input text. Here, the character code is JIS
(2 bytes) code is used. The circles represent the states of the automaton, the arrows represent the state transitions, and the character code attached to each arrow indicates that this transition is triggered. The number written inside each circle indicates the state number of that state. State 0 is the initial state of this automaton. All input character codes for which transitions are not described transit to the initial state 0. An arrow 270 in which a slash "/" is overwritten on the transition represents a transition indicating that "above sea level" is collated. That is, when the transition of the arrow 270 occurs in the state 3, "above sea level" is verified.

The character string collating operation of this prior art will be described below with reference to FIG. The state transition of this automaton starts from the initial state 0. In the initial state 0, if the input character code is “0x33”, the state changes to state 1. Although not shown in the figure, when a character code other than "0x33" is input, the state transits to the initial state 0. Similarly for state 1,
If the input character code is "0x24", transition to state 2; if "0x33", transition to state 1; otherwise return to initial state 0. The same applies to transitions in other states below. In state 3, if the input character code is "0x34", the transition of the arrow 270 in which the collation result is stored occurs and "sea level" is collated.

An example of the state transition table 220 storing the automaton shown in FIG. 4 is shown in FIG.

When the current state number 307 is 0x00 and the value of the input character code 303 is 0x33, 0x01 corresponding to 0x00 and 0x33 is output as the next state number 304 of the automaton to be transited to next. The current state number 307 is 0x03 and the character code is 30.
When 3 is 0x34, the output from the state transition table 220 has a specific value 0xFF indicating that the instructed search term has been collated. Based on this value, the discrimination circuit 260 determines whether or not the instructed search term has been collated. That is, when the output 305 from the state number register 250 is 0xFF, it is determined that the instructed search term has been collated, and the collation result is output.

Output 307 from status number register 214
Is a specific value 0xFE, the contents of the status number register 250 are ignored and the status number register 214 is forced to 0x00.
Is reset to.

FIG. 6 shows a specific example in which this forced reset is required.

The text "mountain and rock" is JIS (2 bytes)
The code is represented by (0x3B33) (0x2448) (0x3464), but it contains the character code (0x3324) (0x4834) representing "above sea level". Therefore, "above sea level", that is, (0x3324) (0x4834)
Character string "mountain and rock" represented by double-byte characters without distinguishing whether the search term appears from the upper byte side or the lower byte side of the double-byte character, that is, (0x3B33)
If (0x2448) (0x3464) is divided into two by 1 byte and the search is performed, the data is collated with a 1-byte shift and an incorrect collation result is output. It is said that the byte shift occurs when the collating process of the character string is started with the lower byte side of the 2-byte character as the initial state.

In order to prevent erroneous collation due to this byte shift, in the prior art, a special value "0xFE" is embedded in the state transition table 220 as shown in FIG. 5, and the discrimination circuit 260 detects "0xFE". I am forced to reset when it is given.

[0021]

As described above, in the document retrieval apparatus according to the prior art, the capacity of the state transition table is large because the 2-byte character is divided into 1 bytes and the collation processing is performed in 1-byte units. However, there is a problem that the processing speed becomes half as compared with the case where the collation process of the 2-byte character is performed once.

In addition, for texts consisting only of double-byte character codes, some measures have been taken to avoid erroneous collation due to byte shifts. However, as in actual Japanese text, single-byte character codes and double-byte character codes are used. You cannot match text that appears with mixed code.

A first object of the present invention is to provide a character code conversion device and a document search device capable of performing collation processing at high speed without enlarging the capacity of the state transition table even for a text composed of double-byte characters. That is.

A second object of the present invention is to provide a character code conversion device and a document search device capable of performing a character string collation process without causing an erroneous collation even in a text in which 1-byte characters and 2-byte characters are mixed. It is to be.

[0025]

All of these problems are to be converted into a predetermined character code system in a bit-compressed form consisting of 2 bytes of a text in which 1-byte characters and 2-byte characters are mixed immediately before the document retrieval device. This is achieved by providing a character code conversion means for

Further, the character code conversion means is two character code acquisition means (hereinafter referred to as a window) for acquiring two bytes from a text in which one-byte characters and two-byte characters are mixed, while shifting them by one byte. Based on the information indicating which window was selected in the previous conversion step and the character type information indicating whether the character code cut out by that window was a 1-byte character or a 2-byte character, the A window in which the first byte is the first byte of a single-byte character or a double-byte character (hereafter,
A character code selection means for cutting out a character code by selecting (a boundary window) and a character type such as whether the character code selected by the character code selection means is a 1-byte character or a 2-byte character. The code conversion means converts the character code according to each character type.

[0027]

In the character code conversion means provided by the present invention, the character code is captured from the input text in two windows in a state where they are shifted by one byte from each other, and the first byte of the captured two bytes is the one-byte character code. Alternatively, the character code is cut out by selecting the character code output from the window that is the upper byte of the 2-byte character code, that is, the window that is the character boundary. In other words, a window loaded from the lower byte side of a 2-byte character (window with byte shift)
By not cutting out the character code output by, the character code can be cut out without causing a byte shift even in the input text in which 1-byte characters and 2-byte characters are mixed.

Further, the cut-out character codes are all code-converted into a predetermined 2-byte character code system in a bit-compressed form in the code conversion unit. As a result, there is no need to have data for an area in the state transition table where the corresponding character code does not exist, so the state transition table can be made compact, and as a result, a fast and inexpensive document retrieval device can be provided. It will be possible.

The detailed principle of the present invention will be described.

FIG. 7 shows an example of fetching character codes in the present invention.

In this figure, the input text is "A and B ...
-", That is, (0x41) (0x82C6) (0
The method of importing a character code will be specifically described by taking the case where x42) (0x82CC) ... Is input as an example.

In this example, the character code is a 1-byte character according to the JIS code system, and a 2-byte character is shifted according to JI.
It is represented by the S (SJIS) code system (hereinafter referred to as the JIS / SJIS code system).

The character codes are fetched in two windows, that is, in two windows, that is, in each of the windows A and B, while being shifted by one byte from each other. That is, in FIG. 7, the first capture takes place in window A (0x41)
And (0x82), but in window B (0x82) and (0xC6)
Is captured.

From the two windows A and B, the output of the window (boundary window) whose first 1 byte is a 1-byte character or the upper byte of a 2-byte character is selected by the character code selector. In this figure, the first 1 byte (0x41) of the previous window, window A, is a 1-byte character, so in the first conversion step, window A is used as the boundary window.
Select. In the second and subsequent conversion steps, which window is the boundary window is determined from the processing result of the conversion step immediately before that. That is, when a 1-byte character is cut out from the window A or a 2-byte character is cut out from the window B in the previous conversion step, the window B becomes the boundary window to be selected next. When a 2-byte character is cut out from the window A or a 1-byte character is cut out from the window B, the window A becomes the boundary window to be selected next.

In the example of this figure, since the character code (0x41) cut out from the window A in the first conversion step is a 1-byte character, the window B is converted in the second conversion step.
Cut out the character code from. Since the character code (0x82C6) cut out from the window B in the second conversion step is a 2-byte character, the character code is cut out from the window B in the third conversion step. In the JIS / SJIS code system, since the character code length is 2 bytes or less, a boundary window appears in at least one of the window A and the window B.

In this way, the character code is always cut out from the first byte of the window while being aware of the byte length, so that the 1-byte character and the 2-byte character can be correctly extracted.

That is, since the character codes are fetched into the two windows while being shifted by 1 byte from each other, and the character code is cut out by selecting the boundary window, the text in which the 1-byte character code and the 2-byte character code are mixed Even with respect to, it is possible to cut out the character code correctly without causing a byte shift.
Then, the character type of the cut-out character code such as 1-byte character or 2-byte character is determined. By performing code conversion according to each character type, the text written in the JIS / SJIS code system is code-converted into a predetermined character code system in a bit-compressed form.

Next, the principle of the character type determination method in the JIS / SJIS code system will be described.

In the JIS / SJIS code system, it is possible to determine whether the cut out character code is a 1-byte character or a 2-byte character by checking the value of the upper 1 byte. That is, when the upper 1 byte is within the range of 0x80 to 0x9F and 0xE0 to 0xEA, the character code can be determined to be a 2-byte character code. If it is in any other range, it can be determined to be a 1-byte character code.

Then, based on the above judgment result, JIS /
A character code represented by the SJIS code system is converted into a predetermined character code system in a bit-compressed form of 2 bytes. FIG. 8 shows a conversion system from the JIS / SJIS code system to a bit-compressed character code system (hereinafter referred to as a compressed character code system).

That is, for the JIS (1 byte) code, it is in the area of 0x0000 to 0x00FF in the character code system after compression conversion, and for the SJIS (2 byte) code, it is 0x0100.
Map to the area of ~ 0x1FA7.

As described above, the capacity of the state transition table can be reduced by converting the input text into the bit-compressed code system using the method described above. The reason will be described below.

The size of the state transition table required for character string matching is generally given by the following equation.

[0044]

[Equation 1] State transition table capacity = (maximum number of states) × (number of character code types) × (number of state number bits) According to this, a double-byte character for a text represented in JIS / SJIS code system The memory capacity required when the state transition table for code is created is expressed by the following equation.

[0045]

## EQU2 ## (256) × (2 ¹⁶ ) × (8) = 128 Mbit = ¹⁶ MByte However, the maximum number of states is 256 (= 2 ⁸ ).

As described above, the memory capacity required for storing the state transition table for the 2-byte character code becomes enormous. The reason for this is that in the JIS / SJIS code system, a region where no corresponding character exists (a region not hatched in FIG. 8A) is extremely wide in the distribution of character codes in a very wide region. Because there are many. As a result, a very large state transition table is required in order to be able to describe state transitions for all character codes.

However, by compressing and converting the input text represented by the JIS / SJIS code system into the code system as shown in FIG. 8B, all the characters can be represented by a 13-bit code. it can.

That is, the memory capacity required when the state transition table is created in the character code system after compression conversion is given by the following equation.

[0049]

Equation 3] (256) × (2 ¹³⁾ × (8) = 16Mbit = 2MByte However, the maximum number of states is 256 (= 2 ^8).

From the above, by performing the compression conversion of the character code, it is possible to reduce the memory capacity of the state transition table, which required 16 MByte, to 2MByte which is ⅛ of that.

As described above, two character code capturing means (windows) for capturing 2 bytes each in a state of shifting by 1 byte and whether or not the window is a boundary window are determined, and 1-byte characters or 2 The capacity of the state transition table can be markedly reduced by providing a character code selection means for cutting out each byte character in character units and a code conversion means for converting this to a bit-compressed character code system, which is inexpensive and high speed. It becomes possible to realize an excellent document retrieval device.

Further, in the JIS / SJIS code system, a 16-bit code was required to represent all characters including Chinese characters, but all characters can be converted by bit-compressed code system. Can be expressed by a 13-bit code, the capacity of data that can be stored in a storage medium such as a magnetic disk device can be equivalently improved by about 20%, and resources can be effectively used. Will be possible. Furthermore, the writing speed and reading speed to the storage medium are equivalently about 20.
%, It becomes possible to effectively use communication means such as lines.

[0053]

EXAMPLE A first example of the present invention will be described below with reference to FIG.

This drawing is a block diagram showing the structure of this embodiment.

In this embodiment, the character code conversion means 400 is provided between the character string storage means 105 and the character string collation means 102, and the 1-byte characters and the 2-byte characters read from the character string storage means 105 are mixed. 2 in a form in which the text 204 is code-compressed by the character code conversion means 400
All the characters are converted into an internal character code system composed of bytes and sent to the character string collating means 102. String matching means 1
In 02, the character code 207 compressed by the character code conversion means 400 is input, and a character string collation process is performed for each character.

The input text of this embodiment is JIS /
It is represented by the SJIS code system.

FIG. 1 is a block diagram showing the configuration of the character code conversion means 400.

The character code conversion means 400 comprises character code acquisition windows 410a and 410b, a character code selection means 420, and a code conversion means 430.

The input texts 204 read from the character string storage means 105 are fetched by 2 bytes each in the character code fetch window A410a and the character code fetch window B410b while being shifted by 1 byte from each other. The character code selection means 420 selects one of the character codes 401 and 402 output from each window whose leading byte is a 1-byte character code or an upper byte of a 2-byte character code. That is, the output of the window whose first byte is the character boundary is selected. That is, the character code is cut out by selecting the character code 401 when the window on the character boundary is A and the character code 402 when the window on the character boundary is B. In the character code 403 cut out by the character code selection means 420, in addition to single-byte characters and double-byte characters, control codes indicating double-width display, half-tone display, etc. (because they are unnecessary for retrieval and are deleted during code conversion, Hereinafter, it is referred to as a deletion code), and binary data such as document number, page number, indent amount, and a specific control code placed immediately before the binary data and indicating that the code following the binary data is binary data (hereinafter referred to as a binary data marker). In addition, control codes having a special meaning such as the end of the document to be searched are mixed.
In the code conversion means 430, the character code selection means 420
These character types are determined for the character code 403 cut out by. Then, by performing code conversion processing according to each character type, code conversion is performed into the 13-bit internal character code system in the bit-compressed form shown in FIG. 8B, and the character string collating means is used as the character code 207 after compression conversion. 102. The flag indicating that the character type information is a 2-byte character is returned to the character code selection means 420 as a 2-byte character flag 404 for determining the boundary window in the next conversion step. .

In this embodiment, a document number is used as binary data mixed in the input text, and a control code having a special meaning is a code indicating the end of the document to be searched (hereinafter referred to as EOF code). To explain. The document number is composed of a 2-byte code following the 1-byte binary data marker.

Next, FIG. 10 shows the configurations of the character code acquisition window A 410a, the character code acquisition window B 410b, and the character code selection means 420 in this embodiment.

First, the character code acquisition window A
The configuration and operation of 410a and B410b will be described.

For example, the text "A and B ...
・ ”, That is, in hexadecimal code representation (0x41) (0x82C6) (0x4
2) The operation when (0x82CC) ... Is input will be described.

First, the first 2 bytes (0x4
182) is taken in.

Of the fetched 2-byte code, the upper 1 byte (0x41) is stored in the upper byte registers 411a and 411b, and the lower 1 byte (0x82) is stored in the lower byte registers 412a and 412b.

Then, when the second input is made,
Stored in upper byte register 411a (0x41)
Is stored in the register 413, the lower byte registers 412a and 412b (0x82) are stored in the registers 414 and 415, respectively. Byte registers 411a and 411b have lower 1 byte
(0x42) is stored in the character code registers 412a and 412b.

The 2-byte code 401 cut out as the window A is the character code registers 413 and 414.
It is composed of the output from, that is, (0x41) and (0x82). The 2-byte code 402 cut out as the window B is the character code registers 415 and 411b.
From (0x82) and (0xC6).

That is, in the present embodiment, the 2-byte code 401 (0x4182) output from the window A and the 2-byte code 402 (0x82C6) output from the window B are shifted by 1 byte from each other. It is taken into the character code selection means 420.

The above is the character code import window 410.
a and 410b.

Next, the character code selection means 420 will be described.

The character code selection means 420 comprises a character code selector 425, an ENOR circuit 423 and a register 424.

In the character code selection means 420, when a 1-byte character is fetched in the window A or a 2-byte character is fetched in the window B in the previous conversion step, the succeeding character code is properly cut out by one character. , The character code captured in the window B in the next conversion step is cut out. In addition, when a double-byte character is captured in window A or a single-byte character is input in window B in the previous conversion step, in the same manner as in the next conversion step, the next succeeding character code is cut out correctly. The character code captured in the window A is cut out.

The operation will be specifically described below.

The character code selector 425 outputs 401 from the character code acquisition window A 410a and 402 from the character code acquisition window B 410b.
Is input, the character code output from the window whose first 1 byte is the upper byte of the 1-byte character or 2-byte character, that is, the boundary window is selected.

This boundary window is a window flag 4 indicating the window selected in the conversion step immediately before it.
21 and a 2-byte character flag 404 indicating the type of character input at that time (whether it is a 1-byte character or a 2-byte character). The boundary window flag 421 indicates that the window A is selected in the previous conversion step when the value is 0, and indicates that the window B is selected when the value is 1. The 2-byte character flag 404 indicates that the input character of the previous conversion step is a 1-byte character when the value is 0, and it is a 2-byte character when the value is 1.

FIG. 11 shows a truth table of a logic for selecting one of the output 401 from the window A and the output 402 from the window B based on the above information.

In the figure, I ₁ is a boundary window flag 421 in the previous conversion step, I ₂ is a 2-byte character flag 404 in the previous conversion step, and I ₀ is a boundary window flag 422 in the current conversion step.

The truth table of this figure is the ENOR circuit 4 of FIG.
23 and a register 424.

Here, the boundary window flag 422 is stored in the register 424, and is referred to as the previous conversion step boundary window flag 421 when the boundary window in the next conversion step is selected.

Next, the specific operation of the character code selection means 420 will be described.

In the character code selector 425, the character code fetched in the window A by selecting the input 401 when the boundary window flag I ₀ is 0, and the input code 40 when the boundary window flag I ₀ is 1.
The character code captured in window B is selected by selecting 2.

1 in window A in the previous conversion step
When byte character is cut out, that is, when I ₁ = 0 and I ₂ = 0, or window when the 2-byte character is incorporated into B, that I ₁ = 1 and the value of I ₀ when the I ₂ = 1 Becomes 1 and the character code selector 425
Selects the input 402 to cut out the character code captured in the window B.

When a double-byte character is fetched in the window A in the previous conversion step, that is, I ₁ =
0 and I ₂ = 1 or when a 1-byte character is input to window B, that is, I ₁ = 1 and I ₂ = 0
Becomes zero the value of I ₀ when the character code selector 4
25 selects window A by selecting input 401
Cut out the character code captured in.

Character code selection means 4 using the example described above.
The specific operation of 20 will be described.

As the initial values, the previous step boundary window flag I ₁ is set to 0 and the previous step character type flag I ₂ is set to 1.

In the first conversion step, I ₁ = 0, I ₂ =
Since 1 and I ₀ = 0, window A is selected, and a 1-byte character (0x41) is cut out from this. In the second conversion step, the character cut out last time is 1-byte code (0x41), so that I ₂ = 0, and I ₁ = 0 to I ₀
Since = 1 is set, the output from window B is selected.
Also, since the 2-byte character (0x82C6) is cut out from the window B in the second conversion step, I ₂ = 1 and I ₁
Since = 1 and I ₀ = 1 are set, the output from the window B is selected in the third conversion step.

Similarly, the character code is cut out correctly from the text in which the 1-byte character and the 2-byte character are mixed while identifying the 1-byte character and the 2-byte character in the same manner.

The above is the detailed operation of the character code selection means 420.

Finally, the code converting means 43 in FIG.
0 will be described.

The code conversion means 430 first determines the character type of the character code 403 cut out by the character code selection means 420. Then, by performing a code conversion process according to each character type, the character code in the input text is converted into the bit-compressed 2-byte internal character code system shown in FIG. 8B.

The code converting means 4 in this embodiment will be described below.
The configuration and operation of 30 will be described.

The structure of the code converting means 430 in this embodiment is shown in FIG.

The code converting means 430 includes a multiplexer 431, a character code converting block 432, a character type determining block 433, an output gate 434, a binary data storage register 435, registers 436 and 437, OR.
It is constituted by the circuit 438.

First, the operations of the multiplexer 431 and the binary data storage register 435 will be described.

The multiplexer 431 selects the character code and the binary data according to the value of the binary data marker flag 439 in the previous processing step. That is, the value 4 of the binary data marker flag in the previous step
When 39 is 1, that is, when the code fetched immediately before is a binary data marker, the following 2 bytes are output to the binary data storage register 435 as a document number here. Then, the binary data stored in the binary data storage register 435 is output to the compound condition judging means 103 as a document number.

When the value 439 of the binary data marker flag in the previous step is 0, that is, when the code fetched immediately before that is a code other than the binary data marker, the subsequent 2-byte code is converted into a character code. The data is output to the block 432 and the character type determination block 433.

The above is the operation of the multiplexer 431 and the binary data storage register 435.

Next, the configuration and operation of the character type determination block 433, the registers 436 and 437, and the OR circuit 438 will be described.

In this embodiment, the character type determination block 433 is composed of a memory called a character type table.

The character type table 433 is referred to by the 2-byte code output from the multiplexer 431 as an address, and the 4-bit flag shown in FIG.
It is composed of a byte character flag, a deletion code flag, a binary data marker flag, and an EOF flag.

That is, the character type table 433 stores a value corresponding to the character type of the code output from the multiplexer 431. Multiplexer 43
When the code output from 1 is a 2-byte character, 1 is output to the 2-byte character flag. Further, 1 is output to the deletion code flag for the deletion code, 1 to the binary data marker flag for the binary data marker, and 1 to the EOF flag for the EOF code.

The 2-byte character flag is stored in the register 436 via the OR gate 438, and is output to the character code selection means 420 as the previous step 2-byte character flag 404 for determining the boundary window in the next step.

The deletion code flag is output to the output gate 43.
4 and the character code conversion block 432 described later.
The output of the conversion character code 440 output from is controlled. That is, when the deletion code flag is 1, the output of the converted character code 440 is suppressed, and when it is 0, the conversion character code 440 is output to the character string collating means 102.

The binary data marker flag is set in the register 4
37, and in the next step the multiplexer 4
31 and the OR circuit 438. Multiplexer 4
In step 31, when the value of the binary data marker flag in the previous step is 1, that is, immediately after the binary data marker is input, the following 2 bytes, that is, the document number is output to the binary data storage register 435. The OR circuit 438 forcibly gives 1 to the code following the binary data marker, that is, the 2-byte character flag corresponding to the binary data, so that the document number is collectively processed as a 2-byte code.

When an EOF code is input, EOF
The flag is output to the search control means 101 as an end signal of the code conversion process.

The above is the operation of the character type determination block 433, the registers 436, 437, and the OR circuit 438.

Finally, the configuration and operation of the character code conversion block 432 and the output gate 434 of FIG. 12 will be described.

In this embodiment, the character code conversion block 432 is composed of a memory called a character code conversion table.

Like the character type table 434, the character code conversion table 432 is referred to by using the 2-byte character code output from the multiplexer 431 as an address. In the table, the values shown in FIG. 39 are stored for each address.

That is, the character code conversion table 432
The character code after the compression conversion for the 2-byte code output from the multiplexer 431 is stored in.
In the output gate 434, when the deletion code flag is 1,
By suppressing the output from the corresponding code conversion block 432, the control code not to be searched is deleted. In the case of a character code other than the deletion code, the value output from the character code conversion table 432 is sent to the character string collating means 102 as it is.

The above is the operation of the character code conversion block 432 and the output gate 433.

As described above, in the present embodiment, the character code conversion means 400 (FIG. 10) is used to determine a plurality of windows that take in 2 bytes each in a state of being shifted by 1 byte and whether or not the window is a boundary window. Judge, 1
Character code selection means for extracting a character code for each character, and character code conversion means for determining a character type for the character code cut out by the character code selection means and performing code conversion processing according to each character type. The character string collating means by correctly extracting the character code from the text in which the 1-byte character and the 2-byte character, and further various control codes are mixed without causing the byte shift, and further performing the code conversion into the compression code system. The capacity of the internal state transition memory can be significantly reduced, and flexible search is possible.

The character code acquisition windows A410a and B410b shown in FIG. 10 can also be realized by the configuration shown in FIG.

That is, in FIG. 10, the upper byte registers 411a and 411b and the lower byte registers 412a and 412b always store the same character code,
Also, the register 414 and the register 415 always store the same character code.

Therefore, by omitting redundant registers in the configuration diagram shown in FIG. 10, the character code acquisition windows 410a and 410b can have a simpler configuration as shown in FIG. That is, FIG.
The register 41 in FIG.
The code corresponding to 1a and the register 411b is stored in the register 416b, and the code corresponding to the registers 412a and 412b is stored in the register 416b. Further, the register 417a in FIG.
The code corresponding to 13 is stored in the register 417b, and the code corresponding to the registers 414 and 415 is stored in the register 417b.

The operation of the character code acquisition windows 410a and 410b in this figure is represented by the text "A".
And B ... ”, that is, in hexadecimal code representation (0x41) (0x
The case where 82C6) (0x42) (0x82CC) ... Is input will be described as an example.

First, the first 2 bytes (0x4
182) is taken in and the upper 1 byte (0x41) is in register 4
16a and the lower 1 byte (0x82) is stored in the register 416b.

Then, when the second input is made,
(0x41) stored in register 416a is register 4
17a, (0x82) stored in the register 416b is stored in the register 417b, and the upper 1 byte (0xC6) of the newly fetched 2 bytes (0xC642) is registered in the register 416.
The lower one byte (0x42) is stored in the register 416b.

The 2-byte code 401 output as the window A is the character code registers 417a and 417.
2-byte code 402 output by b, that is, (0x41) and (0x82), and as window B
Is composed of outputs from the character code registers 417b and 416a, that is, (0x82) and (0xC6). In other words, the windows A and B have taken in the character codes in a state of being shifted by 1 byte from each other.

The above is the operation of the character code capture window shown in FIG.

The character type determination block 433 in the character code conversion means 430 shown in FIG. 12 can also be realized by the configuration shown in FIG.

That is, in the character type table 423 shown in FIG. 12, flags are stored by assigning one bit corresponding to each character type, but in the character type table 433a shown in this figure, the values of these flags are encoded into two bits. Stores the value.

The structure of the character type table value at this time is shown in FIG.
6 shows.

At the time of character type determination, the value of the character type table 433a is decoded by the decoder 433b as shown in FIG. 15 to obtain each character type flag, whereby the capacity of the character type table can be reduced.

Further, the character type determination block 433 in the character code conversion means 430 shown in FIG. 12 can be realized also by the configuration shown in FIG.

Whether it is a 1-byte character or a 2-byte character in the JIS / SJIS code system can be determined by the value of the upper 1 byte of the 2-byte code output from the multiplexer 431. That is, when the value of the upper 1 byte is included in the area of 0x80 to 0x9F and 0xE0 to 0xEA, it can be regarded as a 2-byte character, and in other cases, it can be regarded as a 1-byte character. Therefore, whether the 2-byte code output from the multiplexer 431 is a 2-byte character or not is determined by the comparators 441a, 441b, 441c, and 441d in the circuit shown in this figure.
It is judged by.

For the deletion code, the code values corresponding to the deletion code are previously stored in the registers 442-1 to 442.
n and store them in the comparators 443-1 to 443-
The character code selecting means 420 is compared with the 2-byte code output from the multiplexer 431 at n.
It is possible to determine whether or not the character code 403 cut out by is a deletion code unnecessary for the search.

In addition, a binary data marker and EOF
Similarly for code, binary data marker and EO
The code values corresponding to the F code are respectively registered in the register 444.
And 446 and the comparators 445 and 447.
It is possible to determine whether the character code 403 cut out by the character code selection means 420 is a binary data marker or an EOF code by comparing with the 2-byte code output from the multiplexer 431 at.

As a result, the character type determination circuit 4 shown in FIG.
In 33, 4 × 64 Kbit = 2 for the character type table
While a memory of 56 Kbits was required, the character type determination block shown in this figure does not require a character type table, and therefore character type determination processing is performed, so that the hardware configuration can be made extremely small.

The code converting means 43 shown in FIG.
1, the character code not specified in the search term is deleted, and the number of character codes sent to the character string collating means 102 is reduced, whereby the character string collating means 10
It is possible to reduce the load of 2 and improve the search throughput of the entire document search apparatus.

This will be specifically described below.

In the character type table 432 shown in FIG. 13,
1 is given as the deletion code flag only to the control code (deletion code) that is not the search target. And
When the deletion code flag is 1, the output gate 434 outputs the output 44 from the corresponding code conversion block 432.
The deletion process was performed by gating 0.

Here, 1 is also given as a deletion code flag for a character code not specified in the search term in the character type table 432. Then, like the deletion code, by suppressing the output from the output gate 434 even for the character code that is not specified in the search term,
It becomes possible to perform deletion processing.

Further, in the document retrieval apparatus (FIG. 9) shown in this embodiment, the character code conversion means 400 is different in the JIS / SJIS code system such as half-width full-width characters, uppercase and lowercase letters, katakana hiragana, old and new kanji, etc. By converting the different notation characters represented by the code into the same code in the code system after compression conversion, it becomes possible to collectively search for documents including the different notation characters.

This will be described below by taking the different notation characters (half-width “A” and full-width “A”) of alphanumeric characters as an example.

In the code conversion means 430 shown in FIG. 12, the character code conversion table 432 normally stores the values shown in FIG.

That is, the character code conversion table 432
Stores the character code after the compression conversion for the 2-byte code 403 cut out by the character code selection means 420 (FIG. 1).

Character code conversion table 4 shown in FIG.
In 32, for example, for the address (0x41 ??) (where? Is any code) corresponding to half-width "A" (0x41), (0x004
In 1), (0x01E0) is stored for full-width "A" (0x8260), and half-width "A" and full-width "A" are converted into codes different from each other. That is, when a half-width “A” is used as the search term, the full-width “A” is not matched, and when a full-width “A” is used as the search term, the half-width “A” is matched. Will not be done.

By storing the values shown in FIG. 41 in the character code conversion table 432, it is possible to convert half-width "A" and full-width "A" to the same code.

That is, the address (0x41 ??) corresponding to the half-width "A" and the address (0x826) corresponding to the full-width "A".
By storing the same value (0x01E0) for 0, half-width “A” and full-width “A” can be converted to the same code (0x01E0).

Then, in the character string collating means 102, the retrieval processing is performed with the code after compression conversion (0x01E0) corresponding to the full-width "A". It is possible to search all the displayed characters at once. That is, both "A" and "A" can be searched.

Furthermore, the same method can be applied to uppercase and lowercase letters of English characters, katakana hiragana, and different notation characters of old and new kanji (for example, "A" and "a", "a" and "a", "sai" and "sai"). Thus, it is possible to search all at once by converting to the same compressed and converted code.

However, an exceptional case arises for the different notation characters by half-width and full-width katakana. In other words, the "ba" that is a dull sound is expressed by "ha" (0xCA) and """(0xDE) in the JIS (1 byte) code system.
Full-width "ba" in SJIS (2-byte) code system
Is represented by one 2-byte character (0x836F). As described above, in the half-width character of katakana, the voiced sound and the semi-voiced sound are expressed by two characters of the 1-byte code, but the full-width characters corresponding to these voiced sound and the semi-voiced sound are expressed by the one-byte code of 2 bytes.

Even in such a case, it becomes possible by converting the corresponding different notation characters into the same code, that is, by converting the half-width "ba" and the full-width "ba" into the same compressed and converted code. .

The processing in this case will be described with an example in FIG.

That is, the value of the address of the character code conversion table 432 corresponding to the half-width "ba" is (0xCADE), but the table value corresponding to this is (0x02AF), that is, the table value corresponding to the full-width "ba". Store the same value as (0x02AF). Similarly, for half-width “pa” (0xCADF), the same value (0x02B0) as the table value corresponding to full-width “pa”
To store. Then, the character code (0xCADE) and (0xCAD
F) is treated as a 2-byte character, that is, by setting 1 as a 2-byte character flag corresponding to (0xCADE) and (0xCADF) in the character type table 433, (0xCADE) and (0xCADF) are treated as a 2-byte character. However, it is also possible to search collectively for the different notation characters in half-width and full-width katakana.

The document retrieval apparatus according to the present embodiment has two character code capture windows for capturing two bytes of input text while shifting the input text by one byte, and an output character code from a window forming a character boundary among them. Refer to the character code selection unit that cuts out the character code by selecting, the character type table that gives the character type to the character code cut out by the character code selection unit, and the character code conversion table that gives the converted character code. By converting the input text in which 1-byte characters and 2-byte characters are mixed into a bit-compressed character code system consisting of 2 bytes. ，
Not only can the size of the state transition table in the document retrieval apparatus be made compact, but also flexible and sophisticated retrieval processing can be performed using various control codes embedded in the text. As a result, it is possible to realize an inexpensive, high-speed, and highly functional document retrieval device.

In the first embodiment described above, the character code conversion block 43 in the code conversion means 430 is used.
2 (FIG. 12), code conversion was performed by referring to the character code conversion table that outputs the character code after compression conversion using the 2-byte code output from the multiplexer 431 as an address.

However, in order to perform the code conversion by this method, 64 KW × is used for the character code conversion table.
A memory of 16 bits = 1 Mbit is required.

As a second embodiment of the present invention, in the character code conversion block 432 (FIG. 12) in the first embodiment, instead of the character code conversion table, the upper side 1 of the 2-byte code output from the multiplexer 431 is used. Refer to the code conversion table for 1-byte character that outputs the code conversion value for 1-byte character and the code conversion offset table that outputs the offset information for the code conversion process for 2-byte character by using the byte as an address. An example in which the hardware configuration is made compact by performing code conversion processing using the obtained values will be described.

First, before explaining the configuration and operation of the code converting means in this embodiment, JIS / SJI
A code conversion algorithm in the S code system will be briefly described.

Character code conversion of the SJIS (2-byte) code shown in FIG. 8 can be expressed by the following equation.

(1) When the lower byte is 0x7F or less

[0154]

[Formula 4] SJIS '= (SJIS_H & 0xBF) x 0xBC + SJIS_L-0x5DFC (2) When the lower byte is 0x80 or more

[0155]

SJIS ′ = (SJIS_H & 0xBF) × 0xBC + SJIS_L-0x5DFD In the above formula, SJIS ′ represents the character code after compression conversion, and SJIS_H and SJIS_L are SJI, respectively.
It represents the upper and lower bytes of the S code. The symbol "&" represents a logical product corresponding to bits.

That is, it is determined whether the lower 1 byte of the SJIS (2 bytes) code is larger than 0x80, and if it is smaller, the conversion by the equation 4 is performed, and if it is larger, the conversion by the equation 5 is performed. Code conversion to the compressed character code system shown in (b) can be performed.

The above is the code conversion algorithm in the JIS / SJIS code system.

The structure and operation of the character code conversion block 432 in this embodiment will be described below.

The configuration of the character code conversion block 432 and the character type determination block 433 in this embodiment is shown in FIG.
Shown in.

The character code conversion block 433 in this embodiment includes a 1-byte character code conversion circuit 450, a 2-byte character code conversion circuit 460, and a conversion mode selector 4.
70.

First, the structure and operation of the 1-byte character code conversion circuit 450 will be described.

The 1-byte character code conversion circuit 450 in this embodiment is composed of a memory called a 1-byte character code conversion table.

In the 1-byte character code conversion table, the value 471 of the upper 1 byte of the 2-byte code output from the multiplexer 431 is referred to as an address, and the value shown in FIG. 43 is stored for each address. .

That is, the 1-byte character code conversion table 450 contains the 2 bits output from the multiplexer 431.
The character code after compression conversion for the upper byte 1 byte value 471 of the byte code is stored, and the 1-byte character code conversion is performed by directly outputting the value of the 1-byte character code conversion table.

The above is the code conversion circuit 45 for 1-byte characters.
The content is 0.

Next, the 2-byte character code conversion circuit 46
The configuration and operation of 0 will be described.

The configuration of the 2-byte character code conversion circuit 460 in this embodiment is shown in FIG.

The 2-byte character code conversion circuit 460 in this embodiment is the code conversion offset table 461.
Adder 462, decrementer 463, comparator 46
4, and a selector 465.

Hereinafter, the 2-byte character code conversion circuit 46
The operation of each component of 0 will be specifically described.

First, the code conversion offset table 461 will be described.

The code conversion offset table 461 is 1
Similar to the byte character code conversion table 450, the value 471 of the upper 1 byte of the 2-byte code output from the multiplexer 431 is referred to as an address.

FIG. 44 shows an example of the values stored in the code conversion offset table 461.

That is, the code conversion offset table 461 stores the values shown in the following equation for each address.

[0174]

[Equation 6] OFST = (SJIS_H & 0xBF) × 0xBC-0x5DFC In the above equation, SJIS_H is the value 471 of the upper byte of the 2-byte code output from the multiplexer 431.
, OFST is a code conversion offset table 46.
Represents a table value of 1.

The value shown in Expression 6 is the offset value of the code conversion equation shown in Expression 4, and the code conversion offset table 461 is 2 output from the multiplexer 431.
2 for the value 471 of the upper byte of the byte code
Outputs the offset value for byte character code conversion. That is, the code conversion value for the 2-byte character according to Equation 4 is calculated by adding the value output from the code conversion offset table and the lower byte of the 2-byte character.

Next, adder 462 and dencrementer 46
3, the operations of the comparator 464 and the selector 465 will be described respectively.

In the adder 462, the value output from the code conversion offset table 461 and the multiplexer 43
The 2-byte character code conversion value 475 according to equation 4 is calculated by adding the lower byte 1 byte value 472 of the 2-byte code output from 1.

Also, the decrementer 463 subtracts 1 from the 2-byte character code conversion value 475 of the equation 4 to calculate the 2-byte character code conversion value 476 of the equation 5.

The comparator 464 determines whether the value 472 of the lower 1 byte of the 2-byte code output from the multiplexer 431 is larger than 0x80. Then, when the value 472 of the lower-order 1 byte is smaller than 0x80, the code conversion value 475 by the equation 4 is used, and when it is larger than 0x80, the code conversion value 476 by the equation 5 is used.
2 byte character code conversion value 4 by selecting with
74 is output.

The above is the 2-byte character code conversion circuit 4.
60 operation.

Finally, the operation of the conversion mode selector 470 will be described.

The conversion mode selector 470 selects whether to perform code conversion for 1-byte characters or code conversion for 2-byte characters according to the value of the 2-byte character flag output from the character type determination block 433. That is, when the 2-byte code output from the multiplexer 431 is determined to be a 2-byte character, that is, when the value of the 2-byte character flag is 1, the 2-byte character code conversion value 474 is determined to be a 1-byte character. That is, when the 2-byte character flag is 0, the character code conversion process is performed by selecting the 1-byte character code conversion value 473.

The above is the operation of the character code conversion block 432 in this embodiment.

As described above, in the character code conversion block 432 according to this embodiment, the 1-byte character code for outputting the 1-byte character code conversion value for the upper 1 byte of the 2-byte code output from the multiplexer 431. The conversion process of the character code is performed by referring to the conversion table and the code conversion offset table that outputs the value that is the offset of the code conversion for 2-byte characters. Therefore, in the code converting means shown in the first embodiment of the present invention, the character code conversion table is 64 KW.
In the code conversion means shown in this embodiment, the memory of x16 bit = 1 Mbit is reduced to two memories of 256 W × 16 bit = 4 Kbit for the 1-byte character code conversion table and the code conversion offset table. Therefore, the hardware configuration can be reduced.

The 1-byte character code conversion circuit 450 in this embodiment is composed of a memory called a 1-byte character code conversion table. However, as shown in FIG. 8, since the JIS (1 byte) code is mapped to the area (0x0000) to (0x00FF) in the character code system after compression conversion, the 1-byte character code conversion circuit 4
50 can also be realized by the configuration shown in FIG.

That is, in the 1-byte character code conversion circuit 450 shown in the figure, (0x00) is fetched as the upper 1 byte into the register 451 composed of 2 bytes.
Then, the 1-byte character code conversion value 473 is calculated by taking in the upper-order 1 byte of the 2-byte code output from the multiplexer 431 as the lower 1 byte.

In this way, by shifting the high-order 1 byte to the low-order byte and calculating the 1-byte character code conversion value, the 1-byte character code conversion circuit 450 shown in FIG. The conversion table, which requires a memory of 256 W × 16 bits = 4 Kbits, can calculate the 1-byte character code conversion value 473 without using the memory, and the hardware configuration of the 1-byte character code conversion unit. Can be reduced.

In the character code conversion block 432 shown in FIG. 18, the 1-byte character code conversion circuit 45 is used.
Both the 1-byte character code conversion table and the code conversion offset table 461 in the 2-byte character code conversion circuit 460 that form 0 are multiplexer 431.
1 byte 471 of high-order side of 2-byte code output from
Is referred to as an address. Furthermore, JIS / SJI
In the S code system, the area where the 1-byte character code exists (0x00-0x7F, 0xA0-0xDF) and the area where the upper 1-byte code of the 2-byte character code exists (0x80-0x9F, 0xE)
0 to 0xFF) are in mutually exclusive areas. Therefore, FIG.
Area (0x00-0x7F, 0xA0-0xDF) in which the 1-byte character code conversion table 450 has a value and area in which the code conversion offset table 471 in FIG. 44 has a value
(0x81 to 0x9F, 0xE0 to 0xEA) are of course mutually exclusive areas.

From the above, in the character code conversion block 432 shown in FIG. 18, the contents of the 1-byte character code conversion table forming the 1-byte character code conversion circuit 450 are stored in the 2-byte character code conversion circuit 460. Can also be held in the code conversion offset table 461 of

At this time, the character code conversion block 43
The configuration of No. 2 is shown in FIG.

In the character code conversion block 432 shown in this figure, the code conversion offset table 461 is used.
Stores the values shown in FIG.

That is, in the code conversion offset table 461 shown in this figure, the address 471 is included in the 1-byte character code area (0x00-0x7F, 0xA0-0xDF), that is, the upper side of the 2-byte code output from the multiplexer 431. When the 1-byte code 471 is a 1-byte character code, the 1-byte character code conversion value 473 is output. When the address 471 is included in the upper byte area (0x80 to 0x9F, 0xE0 to 0xFF) of the 2-byte character, a value that is an offset for the 2-byte character code conversion process is output.

Further, the adder 462 and the decrementer 46
3, the comparator 464, the selector 465, and the conversion mode selector 470, like the 2-byte character code conversion circuit 460 shown in FIG. 19, are offsets for the 2-byte character code conversion processing output from the code conversion offset table 461. Value and multiplexer 431
Value of 1 byte of lower side of 2 byte code output from
A 2-byte character code conversion value 474 is calculated from 72.

As described above, the code conversion offset table 461 in the 2-byte character code conversion circuit 460 also has the contents of the 1-byte character code conversion table constituting the 1-byte character code conversion circuit 450. In the character code conversion block shown in FIG. 18, it is possible to reduce one that requires two 256W × 16bit = 4Kbit memories to one 256W × 16bit = 4Kbit memory, and to reduce the hardware configuration. It will be possible.

In the second embodiment, the input text represented by the JIS / SJIS code system was code-converted into a bit-compressed code system with no gaps. That is, as can be seen from FIG. 8, in the SJIS code system, the area where the character code of the lower byte is 0x7F is not mapped and the code conversion is performed to the compressed code system without any gap. 2-byte character code conversion circuit 460 (FIG. 19)
The configuration of will become complicated.

As a third embodiment of the present invention, a simplified configuration of the 2-byte character code conversion circuit 460 in the second embodiment by partially allowing a gap in the compression code system will be described.

First, the conversion algorithm of the character code in this embodiment will be described.

In this embodiment, the 2-byte character code represented by the SJIS code system is converted by the conversion formula shown below.

[0199]

22 shows the correspondence between SJIS ″ = (SJIS_H & 0xBF) × 0xBD + SJIS_L-0x5E7D JIS / SJIS code system and the character code after compression conversion by the above conversion formula.

As can be seen from the figure, in the compressed code system, there is an area where the character code is not assigned in the area corresponding to the area where the lower byte in the JIS / SJIS code system is 0x7F. (2 bytes) Character code is 0x0100-0x1FA7
The code is converted into the area of, and all characters can be expressed by a 13-bit code.

The above is the code conversion algorithm in this embodiment.

Next, the configuration and operation of the 2-byte character code conversion circuit 460 when the code conversion processing based on the equation 7 is performed will be described.

The structure of the 2-byte character code conversion circuit 460 in this embodiment is shown in FIG.

The 2-byte character code conversion circuit 460 shown in the figure comprises a code conversion offset table 461 and an adder 462.

First, the code conversion offset table 461 will be described.

In the code conversion offset table 461, the value 471 of the upper 1 byte of the 2-byte code output from the multiplexer 431 is referred to as an address, and the value shown in the following equation is stored in each address.

[0207]

Ofst = (SJIS_H & 0xBF) × 0xBD-0x5E7D In addition, the code conversion offset table 461 at this time
The value of is shown in FIG.

The value shown in Expression 8 is the offset value of the code conversion expression shown in Expression 7, and the 2-byte character code conversion value 474 is the value output from the code conversion offset table 461 and the 2-byte character. Value 4 in lower 1 byte
72 (SJIS_L) is added by the adder 462 to easily calculate.

As described above, in the 2-byte character code conversion circuit 460 shown in FIG. 23, a gap is partially allowed in the character code system after compression conversion, so that FIG.
The decrementer 463, the comparator 464, and the selector 465 can be omitted as compared with the 2-byte character code conversion circuit 460 shown in (4), and the hardware configuration can be reduced.

In addition, in the second embodiment, 2 shown in FIG.
By referring to a code conversion offset table that outputs a value that is an offset for 2-byte character code conversion, using the upper 1 byte 471 of the 2-byte code output from the multiplexer 432 as the byte character code conversion circuit 460 as an address. Code conversion is performed.

However, in order to perform character code conversion by this method, 4K for the code conversion offset table is used.
Bit memory is required.

As the fourth embodiment of the present invention, the code conversion offset table 461 is not used in the 2-byte character code conversion circuit 460 in the code conversion means 430 in the second embodiment, and the code conversion is performed by the arithmetic processing. What you do is explained.

The configuration and operation of the 2-byte character code conversion circuit 460 in the fourth embodiment will be described below.

First, the configuration of the 2-byte character code conversion circuit 460 in this embodiment is shown in FIG.

In the 2-byte character code conversion circuit 460 shown in this figure, the 2-byte character code conversion value is calculated in accordance with equations (4) and (5). That is, the value obtained by bitwise ANDing the value 471 (SJIS_H) of the high-order 1 byte and 0xBF is multiplied by 0xBC, and this is multiplied by the value 47 of the low-order 1 byte 47.
Intermediate result 477 by adding 2 (SJIS_L)
Ask for. Further, the conversion result 475 is calculated by subtracting 0x5DFC from the intermediate result 477. Further, the conversion result 476 is calculated by subtracting 0x5DFD from the intermediate result 477. That is, in the figure, the conversion result 475 indicates the code conversion value by the equation 4, and the conversion result 476 indicates the code conversion value by the equation 5.

In the comparator 464, the multiplexer 4
The lower 1 byte of the 2-byte code output from 31 is
By determining whether or not it is larger than 0x80, it is determined whether the conversion by the equation 4 or the conversion by the equation 5 should be performed. That is, when the value 472 of the lower 1 byte is smaller than 0x80, the selector 465 outputs the conversion result 475,
When it is larger than 0x80, the conversion result 476 is selected and output as the 2-byte character code conversion value 474.

The above is the code conversion circuit 4 for 2-byte characters.
60 operation.

As described above, in the 2-byte character code conversion block 432 shown in this embodiment, the character code conversion processing is performed by the arithmetic processing. Therefore, the 2-byte character code conversion circuit 460 (FIG. 19) shown in the second embodiment of the present invention uses a memory of 256 W × 16 bits = 4 Kbits as a code conversion offset table. It is possible to perform code conversion processing without having to do so, and it is possible to reduce the hardware configuration.

Also in this embodiment, as in the third embodiment, the structure of the 2-byte character code conversion value calculation circuit 470 can be simplified by partially allowing a gap in the code system after compression conversion. it can. FIG. 25 shows a configuration diagram of the code converting means 430 in this case.

In the 2-byte character code conversion circuit 460 shown in this figure, the 2-byte code conversion value is calculated according to the 2-byte character code conversion formula shown in Eq. That is, the value 471 (SJIS_H) of the upper 1 byte and 0xBF
Multiply by the value 481 which is the bit logical product of and and 0xBC.
From this value 472, the value 472 of 0x5E75 and the lower 1 byte
A 2-byte character code conversion value 474 is obtained by subtracting the value 483 obtained by subtracting (SJIS_L). In this embodiment, the hardware configuration can be made smaller than that of the 2-byte character code conversion circuit shown in FIG.

In the above embodiment, JIS / SJIS
The configuration and operation of the character code conversion means when the document represented by the code system is the search target have been described. However, if the character code is 1-byte character, EBCDI
K code system, 1-byte character and 2-byte character determination method and compressed code system for documents represented by KEIS code system for double-byte characters (hereinafter referred to as EBCDIK / KEIS code system) Since the conversion algorithm for the is different, a character code conversion means with a different structure is required.

As a fifth embodiment, EBCDIK / KE
The configuration and operation of the code conversion means when a document represented by the IS code system is targeted will be described with an example.

First, before explaining the configuration and operation of the code converting means in this embodiment, EBCDIK /
A method for determining a 1-byte character and a 2-byte character in the KEIS code system and a conversion algorithm to the compression code system will be briefly described.

First, a method of determining a 1-byte character and a 2-byte character in the EBCDIK / KEIS code system will be described.

In the EBCDIK / KEIS code system, the stage (hereinafter,
There is a stage in which only 2-byte characters are input (hereinafter referred to as a 1-byte stage) (hereinafter referred to as a 2-byte stage). Then, the 1-byte stage and the 2-byte stage are separated by a specific control code (hereinafter referred to as an escape sequence). That is, when the control code (0x0A41) indicating the transition to the 1-byte stage is input, the control code indicating the transition to the 2-byte stage is input next.
It becomes a 1-byte stage and only 1-byte characters are input until (0x0A42) is input. When a control code (0x0A42) indicating a transition to the 2-byte stage is input, a control code (0x0A4) indicating a transition to the 1-byte stage is input next.
It becomes a 2-byte stage until 1) is input and only 2-byte characters are input. In other words, it is currently determined whether the input character code is a 1-byte character or a 2-byte character.
It is determined by whether it is a byte stage or a 2-byte stage, which is performed by detecting an escape sequence.

The above is a method for determining a 1-byte character and a 2-byte character in the EBCDIK / KEIS code system.

Next, the conversion algorithm from the EBCDIK / KEIS code system to the compressed code system will be described.

FIG. 26 shows a conversion system from the EBCDIK / KEIS code system to the compressed code system.

That is, for the EBCDIK (1 byte) code, 0x0000 to 0x in the character code system after compression conversion.
00x area, 0x0100-0x1F for KEIS code
Map to 7D area.

The above is the conversion algorithm from the EBCDIK / KEIS code system to the compressed code system.

As described above, in the document represented by the EBCDIK / KEIS code system, the character type determination method and the conversion method to the character code system after compression conversion are JI
Since it is different from the one expressed in the S / S JIS code system,
A code conversion block 432 and a character type determination block 433 having different configurations are required.

The code conversion block 43 in this embodiment.
FIG. 27 shows the configuration of the 2 and character type determination block 433.

In this embodiment, the code conversion block 4
32 and a character type determination block 433 are an escape sequence detection circuit 510, a character type determination circuit 520 for 1-byte characters, a character type determination circuit 530 for 2-byte characters, a determination mode selector 540, a code conversion circuit 450 for 1-byte characters, a code for 2-byte characters. It is composed of a conversion circuit 460 and a conversion mode selector 470.

The operations of the code conversion block 432 and the character type determination block 433 in this embodiment will be described below.

First, the operation of escape sequence detection circuit 510 will be described.

The escape sequence detection circuit 510 detects the escape sequence from the 2-byte code output from the multiplexer 431 to determine whether it is the 1-byte stage or the 2-byte stage, that is, the input character is 1 Is it a byte character or 2
Determine if it is a byte character.

The configuration of the 2-byte character determination circuit 510 is shown in FIG.
8 shows.

In the comparator 511, the multiplexer 4
It is determined whether the 2-byte code output from 31 is an escape sequence to the 1-byte stage, and the comparator 512 determines whether it is an escape sequence to the 2-byte stage. The flip-flop 513 determines from the outputs of the comparators 511 and 512 whether it is a 1-byte stage or a 2-byte stage. That is, when the escape sequence to the 2-byte stage is input, 1 is output from the comparator 512, and by setting the value of the flip-flop 513 to 1, the transition to the 2-byte stage is shown.
Further, when the escape sequence to the 1-byte stage is input, 1 is output from the comparator 511, which indicates that the value of the flip-flop 513 is reset and the transition to the 1-byte stage is made. Also, O
In the R circuit 514, since the escape sequence itself is treated as a 2-byte code, the output from the flip-flop 513 and the output from the comparators 511 and 512 are ORed.
The double-byte character flag is set by taking.

The above is the operation of the 2-byte character determination circuit 510.

Next, the character type determination circuit 52 for 1-byte characters
The operation of the character type determination circuit 530 for 0 and 2-byte characters and the determination mode selector 540 will be described.

In the character type determination circuit 520 for 1-byte characters, the upper byte of the 2-byte code output from the multiplexer 431 is input to the EBCDI.
The character type in the K (1 byte) code system is determined, that is, whether it is a deletion character, whether it is a binary data marker, or whether it is an EOF code.

Also, the character type determination circuit 53 for double-byte characters
At 0, the 2-byte code output from the multiplexer 431 is used as an input to determine the character type in the KEIS (2-byte) code system.

In the present embodiment, the 1-byte character type determination circuit 520 and the 2-byte character type determination circuit 530 are respectively a 1-byte character type determination table,
It is configured by a table called a character type determination table for double-byte characters. The character type determination table for 1-byte characters shown in FIG. 29 has the upper 1-byte code of the 2-byte code output from the multiplexer 431 as the address.
It outputs a bit flag, that is, a deletion character flag, a binary code marker flag, and an EOF flag. Also, 2
Also in the character type determination table for byte characters, the 2-bit code output from the multiplexer 431 is used as an address and the 3-bit flag shown in FIG. 30 is stored.

The determination mode selector 540 determines the character type by selecting the appropriate value from the output from the character type determination circuit 520 for 1-byte characters and the output from the character type determination circuit 530 for 2-byte characters. That is, 2
When the value of the byte character flag is 0, the output from the 1-byte character character type determination circuit 520 is output, and when the value of the 2-byte character flag is 1, the 2-byte character character type determination circuit 53.
Select output from 0.

The above is the character type determination circuit 5 for 1-byte characters.
These are the operations of the character type determination circuit 530 for 20 and 2-byte characters and the determination mode selector 540.

Finally, the 1-byte character code conversion circuit 4
The operations of the 50 and 2-byte character code conversion circuit 460 and the conversion mode selector 470 will be described.

In this embodiment, the 1-byte character code conversion circuit 450 and the 2-byte character code conversion circuit 4 are used.
Reference numerals 60 are constituted by tables called a 1-byte character code conversion table and a 2-byte character code conversion table, respectively.

In the 1-byte character code conversion table, the upper 1-byte code of the 2-byte code output from the multiplexer 431 is used as an address, and the value shown in FIG. 47 is stored in each address.

That is, the character code after compression conversion for each address is stored in the 1-byte character code conversion table, and the conversion processing for 1-byte character outputs the value of the 1-byte character code conversion table as it is. It is done by

In the 2-byte character code conversion table, the 2-byte code output from the multiplexer 431 is used as an address, and the value shown in FIG. 48 is stored in each address.

Similarly to the 1-byte character code conversion table, the 2-byte character code conversion table stores the character code after compression conversion for each address.
The conversion process for 2-byte characters is performed by directly outputting the values in the 2-byte character code conversion table.

In the conversion mode selector 470, when the input character code is a 1-byte character, that is, when the 2-byte character flag is 0, the output from the 1-byte character code conversion circuit 450 is output, and when it is a 2-byte character, that is, 2 bytes. When the character flag is 1, a corresponding code conversion value is given by selecting an output from the 2-byte character code conversion circuit 530.

The above is the code conversion circuit 4 for 1-byte characters.
These are the operations of the 50 and 2-byte character code conversion circuit 460 and the conversion mode selector 470.

In the present embodiment, EBCDIK /
The code conversion processing of the text represented by the KEIS code system was realized by referring to the 1-byte character code conversion table and the 2-byte character code conversion table, respectively. The values stored in these tables and JIS
By associating the character code after the compression conversion in the / SJI code system with each other, the texts represented by these two different code systems can be converted into the same compressed code system.

In the present embodiment, the 1-byte character character type determination circuit 520 and the 2-byte character character type determination circuit 530 are the 1-byte character character type determination table and the 2-byte character type determination circuit.
Although it is configured by the character type determination table for byte characters, it is obvious that it can be configured by the decoder as shown in the first embodiment. Further, the 1-byte character code conversion circuit 450 and the 2-byte character code conversion circuit 460 are composed of a 1-byte character code conversion table and a 2-byte character code conversion table. It is also apparent that the hardware configuration can be simplified by reducing the capacity of the table as shown in the fourth embodiment.

As described above, by providing the escape sequence detection circuit for determining whether it is the 1-byte stage or the 2-byte stage in the character type determination block in the code conversion means, the EBCDIK / KEIS code system is used. The character code can be compressed and converted for a document.

In the above embodiment, JIS / SJIS
The configuration and operation of the character code conversion means for the document represented by the code system and the EBCDIK / KEIS code system have been described. However, for a document whose character code is represented by the JIS code system for 1-byte characters and the JIS code system for 2-byte characters (hereinafter referred to as JIS / JIS code system), a 3-byte character code is used. Since the control code represented by the code is included, the configuration of the character code capture window and the conversion algorithm to the compressed code system are different, and thus the configuration of the character code conversion means is different.

As a sixth embodiment, the structure and operation of the code conversion means when a document represented by the JIS / JIS code system is used will be described by way of example.

First, before explaining the configuration and operation of the code converting means in this embodiment, JIS / JIS
A method of determining a 1-byte character and a 2-byte character in the code system and a conversion algorithm to the compressed code system will be briefly described.

First, a method of determining a 1-byte character and a 2-byte character in the JIS / JIS code system will be described.

In the JIS / JIS code system, 1
The byte stage and the 2-byte stage are separated by a 3-byte escape sequence. That is, 1
Escape sequence indicating transition to byte stage
When (0x1B284A) is input, the 1-byte stage is entered and only 1-byte characters are input until the next escape sequence (0x1B2442) indicating the transition to the 2-byte stage is input. When an escape sequence (0x1B284A) indicating a transition to the 2-byte stage is input, the next 1
Escape sequence indicating transition to byte stage
It becomes a 2-byte stage until (0x1B2442) is input. 2
Only byte characters are entered. That is, whether the input character code is a 1-byte character or a 2-byte character is determined depending on whether it is currently the 1-byte stage or the 2-byte stage, which is performed by detecting the escape sequence.

The above is the method for determining a 1-byte character and a 2-byte character in the JIS / JIS code system.

Next, a conversion algorithm from the JIS / JIS code system to the compressed code system will be described. J
FIG. 31 shows a conversion method from the IS / JIS code system to the compressed code system.

That is, for the JIS (1 byte) code, it is in the area of 0x0000 to 0x00FF in the character code system after compression conversion, and for the JIS (2 byte) code, it is 0x0100 to
Map to the area of 0x1F7D.

The above is the conversion algorithm from the JIS / JIS code system to the compressed code system.

As described above, in a document represented by the JIS / JIS code system, one-byte characters, two-byte characters, and three-byte characters appear in a mixed manner, so that the code conversion means 400 having a different structure is required. .

Character code conversion means 40 in this embodiment
The configuration of 0 is shown in a block diagram in FIG.

The configuration of the character code conversion means 400 is shown in a block diagram in FIG.

The character code conversion means 400 comprises a bit width conversion means 710, a character code acquisition window 720a,
720b and 720c, character code selection means 420,
The code conversion means 430 is used.

The input text 204 read out from the character string storage means 105 16 bits at a time is the bit width conversion means 7
Converted to a width of 24 bits by 10. Then, the character code acquisition window A720a and the character code acquisition window B72 are shifted by 1 byte from each other.
3 bytes are captured in 0b and the character code capture window c. In the character code selection means 420, the character codes 702, 703 and 7 output from each window are output.
From 04, the first byte is a 1-byte character code, the upper byte of a 2-byte character code, or the uppermost byte of a 3-byte character code, that is, the output of a window in which the first byte is a character boundary is selected.

The code conversion means 430 determines the type of these characters for the character code 403 cut out by the character code selection means 420. Then, by performing code conversion processing according to each character type,
Character code 2 after code conversion to the 13-bit internal character code system in the bit-compressed form shown in (b) and compression conversion
It is sent to the character string collating means 102 as 07. Further, among the character type information determined here, the flag indicating that it is a 2-byte character and the flag indicating that it is a 3-byte character are each a 2-byte character in order to determine the boundary window in the next conversion step. Flags 404 and 3
As the byte character flag 705, the character code selection means 42
Returned to 0.

Next, the structure and operation of the bit width conversion means 710 will be described.

The structure of the bit width conversion means 710 is shown in FIG.

For example, as text, "A and B ...",
That is, in hexadecimal code expression (0x41) (0x1B2442) (0x2448)
The operation when (0x1B284A) (0x41) ... Is input will be described.

[0275] First, the first 2 bytes (0x4
11B) is taken in.

Of the fetched 2-byte codes, the upper 1 byte (0x41) is the upper byte register 711a.
The lower 1 byte (0x1B) is the lower byte register 71
It is stored in 1b.

Then, when the second input is made,
Stored in upper byte register 711a (0x41)
Is in the register 712a and the lower byte register 711b
(0x1B) is stored in the register 712b, and the newly captured 2 bytes (0x2442) of the upper byte 1 byte (0x24) are stored in the upper byte register 711a and the lower byte (0x42) is stored in the character string. It is stored in the code register 711b.

The selector 715 has registers 712a, 71a.
3-byte code 71 output from 2b and 711a
3 and the 3-byte code 714 output from the registers 712b, 711a, and 711b are input, and these are alternately selected to convert the character code input in 16-bit width into 24-bit width.

In other words, in this example, the selector 715 has 3
(0x411b24) is input as the byte code 713, (0x1b2442) is input as the 3-byte code 714, and the 3-byte code 713 is selected as the initial condition (0x411b24)
Is output.

When the third input is made, it is stored in the upper byte register 711a (0
x24) is in the register 712a and the lower byte register 41
1b (0x42 is stored in the register 712b and newly captured 2 bytes (0x2448) upper side 1
The byte (0x24) is stored in the upper byte register 711a, and the lower one byte (0x48) is stored in the character code register 711b.

At this time, the selector 715 stores (0x244224) as the 3-byte code 713 and the 3-byte code 714.
Is input as (0x422448). In the selector 715,
Since the 3-byte code 713 side was selected in the previous processing step, (0x422448) is output by selecting the 3-byte code 714 in this processing.

The above is the operation of the bit width conversion means 710.

Next, a character code acquisition window A72
0a, the character code acquisition window B720b, and the character code acquisition window C720c are shown in FIG. 34, and their operations will be described.

[0284] First, the character code acquisition window A
720a, character code acquisition window B720b and character code acquisition window C720c are shown in FIG.
4 shows.

[0285] First, the first 3 bytes (0x4
11b24) is imported.

1 out of the fetched 3-byte code
Byte (0x41) is upper register 721a, 721b
And 721c, the second byte (0x1b) is stored in the middle registers 722a, 722b and 722c, and the third byte is stored.
(0x24) is lower register 723a, 723b and 72
3c is stored.

Then, when the second input is made,
(0x41) stored in the upper register 721a is stored in the register 724, and the middle registers 722a and 722b are stored.
(0x1b) is stored in registers 725 and 727, and (0x24) stored in lower registers 723a, 723b and 723c is stored in register 7 respectively.
The first byte (0x42) of the newly captured 3 bytes (0x422448) stored in Nos. 26, 728 and 729 is in the upper registers 721a, 721b and 721c, and the second byte (0x24) is in the middle register 722a, 722b and 7
22c, the third byte (0x48) is lower register 723
a, 723b and 723c.

The 3-byte code 702 cut out as the window A is the registers 724, 725, and 726.
From (0x41), (0x1b) and (0x24). Also, it is cut out as window B 3
Bytecode 703 is registered in registers 727, 728 and 7
The output from 21b, that is, (0x1b), (0x24), and (0x42), the 3-byte code 704 cut out as the window C is the output from the registers 729, 721c, and 722c, that is, (0x24), (0x42), and (0x24). ).

That is, in this embodiment, the 3-byte code 702 (0x411b24) output from the window A and the 3-byte code 703 (0x1b244) output from the window B are output.
2), and 3-byte code 7 output from window B
04 (0x244224) is taken into the character code selection means 420 in a state where they are shifted by 1 byte from each other.

The above is the character code import window 720.
a, 720b and 720c.

Next, the character code selection means 420 will be described.

FIG. 35 shows the configuration of the character code selection means 420.
Shown in.

In the character code selection means 420, as in the first embodiment, information indicating from which window the character code was cut out in the previous conversion step and whether the character cut out by the window is a 1-byte character or 2 bytes. A process of determining and selecting a window for cutting out a character code in the current conversion step is performed based on information such as whether it is a character or a 3-byte character.

Which window the character code fetched into is selected is determined by the values of the window A attention flag 733, the window B attention flag 734 and the window C attention flag 735.

That is, these flags are mutually exclusive. In the character code selector 731, the character code 702 taken into the window A when the window A attention flag 733 is 1 and the window B attention flag 734 is 1 in the character code selector 731. The character code 703 fetched in the window B is the character code 704 fetched in the window C when the window C attention flag 735 is 1.
The character code is cut out by selecting.

Also, in the decoder 732, the previous step 2-byte character flag 739 indicating that the character code fetched in the previous step is a 2-byte character, and the character code fetched in the previous step is a 3-byte character Previous step 3 byte character flag 740, previous step window A attention flag 741, previous step window B attention flag 742, previous step window C attention flag 7
43, from which window the character code is to be cut out in the current step, that is, the window A focus flag 73
3, window B attention flag 734 and window C
The value of the attention flag 735 is determined.

For example, in the previous step, window A
When a 1-byte character code is fetched into, that is, the previous step window A focused window flag 741 is 1
When both the previous step two-byte character flag 739 and the previous step three-byte character flag 740 are 0, the character code is cut out from the window that is shifted by one byte from the window A, that is, the window B. Also, when a 2-byte character code is fetched in window A, that is, the previous step window A focused window flag 741
Is 1 and the previous step 2-byte character flag 739 is 1, the character code is cut out from the window shifted from the window A by 2 bytes, that is, the window C.

As described above, in the decoder 732, as shown in FIG.
According to the truth table shown in FIG.
3, window B attention flag 734 and window C
The focus flag 735 is output.

Character code selection means 4 using the example described above
The specific operation of 20 will be described.

As an initial value, the window A attention flag 7
41 is set to 1, the window B attention flag 742 is set to 0, and the window C attention flag 743 is set to 0.

In the first conversion step, since the value of the window A focus flag 741 is 1, window A is selected, and a 1-byte character (0x41) is cut out from this. Two
In the conversion step of the first time, a 1-byte character (0x41) was cut out from the window A in the previous step, that is, the previous step window A attention flag 741 is 1, and the previous step 2-byte character flag 739 and the previous step 3-byte character flag 740 Since both are 0, the window B focus flag 734 becomes 1 and the 3-byte character (0x1B2
442) is cut out. Also, in the third conversion step, 3-byte characters from window B (0x1B2442) in the previous step
Is cut out, that is, the previous step window B attention flag 742 is 1, the previous step 2 byte character flag 739 is 0, the previous step 3 byte character flag 740 is 1, so the window B attention flag 734 is 1 and the window B The 3-byte character (0x1B2442) is cut out.

In the same manner, the character code is correctly cut out from the text in which the 1-byte character, the 2-byte character and the 3-byte character are mixed while identifying the 1-byte character, the 2-byte character and the 3-byte character in the same manner.

The above is the details of the operation of the character code selection means 420.

Finally, the code converting means 430 will be described.

The configuration of the code converting means 430 in this embodiment is shown in FIG.

In the code conversion means 430 of this embodiment, the escape sequence is detected by the 24-bit value output from the character code selection means in the character type determination block 433, and if the escape sequence is detected, the previous step is carried out. 1 is output to the 3-byte character flag 705. Therefore, the configuration of the escape sequence detection circuit in the character type determination block 433 (FIG. 27) in the fifth embodiment is different.

FIG. 38 shows the configuration of the escape sequence detection circuit in this embodiment.

The comparator 511 determines whether or not the 3-byte code output from the character code selection means 431 is an escape sequence to the 1-byte stage, and the comparator 512 determines whether it is an escape sequence to the 2-byte stage. . Flip-flop 51
At 3, the output from the comparators 511 and 512 determines whether the stage is a 1-byte stage or a 2-byte stage. The OR circuit 515 ORs the output from the comparator 511 and the output from the comparator 512 to generate a 3-byte character flag. The AND circuit 516 generates a 2-byte character flag on the condition that it is a 2-byte stage and is not an escape sequence.

The above is the operation of the escape sequence detection circuit 510 in this embodiment.

In the present embodiment, the 1-byte character character type determination circuit 520 and the 2-byte character character type determination circuit 530 are respectively a 1-byte character character type determination table and a 2-byte character character type as in the fifth embodiment. It is realized by the judgment table.

Also, the 1-byte character code conversion circuit 45
The 0- and 2-byte character code conversion circuit 460 is also realized by the 1-byte character code conversion table and the 2-byte character code conversion table, respectively, as in the fifth embodiment.

The values shown in FIG. 49 are stored for each address in the 1-byte character code conversion table.

Further, the 2-byte character code conversion table stores the values shown in FIG. 50 for each address.

In this embodiment, JIS / JIS
The code conversion processing of the text represented by the code system was realized by referring to the code conversion table for 1-byte character and the code conversion table for 2-byte character, respectively. The values stored in these tables and JIS / SJI
By associating with the character code after the compression conversion in the code system, the text represented by these two different code systems can be converted into the same compressed code system.

In the present embodiment, the 1-byte character character type determination circuit 520 and the 2-byte character character type determination circuit 530 are a 1-byte character character type determination table and a 2-byte character type determination circuit.
Although it is configured by the character type determination table for byte characters, it is obvious that it can be configured by the decoder as shown in the first embodiment. Further, the 1-byte character code conversion circuit 450 and the 2-byte character code conversion circuit 460 are composed of a 1-byte character code conversion table and a 2-byte character code conversion table. It is also clear that the hardware configuration can be simplified by reducing the capacity of the table as shown in the embodiment.

As described above, by providing the escape sequence detection circuit for detecting the 3-byte escape sequence in the character type determination block in the code conversion means, the character code can be applied to the document represented by the JIS / JIS code system. Can be compressed and converted.

[0317]

EFFECTS OF THE INVENTION Two character code capturing means (windows) for capturing 2 bytes at a time by shifting by 1 byte
And a character code selection means for judging whether or not the window is a boundary window and cutting out a 1-byte character or a 2-byte character in character units, and a code conversion means for converting this to a bit-compressed character code system. As a result, the capacity of the state transition table can be significantly reduced, and even in texts in which 1-byte characters and 2-byte characters are mixed, high-speed character string matching can be performed without causing erroneous matching due to byte shifts. It becomes possible to realize a simple character string search device.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing an embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of a conventional character string search device.

FIG. 3 is a block diagram showing a configuration of a reference character string collating means.

FIG. 4 is a diagram showing an example of an automaton for matching search terms.

FIG. 5 is a diagram showing an example of contents of a state transition table used in a reference.

FIG. 6 is a diagram showing an example in which erroneous collation occurs due to a byte shift.

FIG. 7 is a diagram showing an example of a character code capturing method according to the present invention.

FIG. 8 is a diagram showing an example of a conversion format from the SJIS code system to the compressed character code system.

FIG. 9 is a diagram showing a configuration of an embodiment of a character string search device according to the present invention.

FIG. 10 is a configuration diagram showing an embodiment of a character code acquisition window and character code selection means according to the present invention.

FIG. 11 is a truth table describing the operation of one embodiment of the character code selection means according to the present invention.

FIG. 12 is a configuration diagram showing an embodiment of a code converting means according to the present invention.

FIG. 13 is a diagram showing an embodiment of a character type determination block in the present invention.

FIG. 14 is a diagram showing an embodiment of a simplified character code acquisition window according to the present invention.

FIG. 15 is a diagram showing an embodiment of a character type determination block according to the present invention.

FIG. 16 is a diagram showing an embodiment of a simplified character type determination block according to the present invention.

FIG. 17 is a diagram showing an embodiment of a character type determination circuit according to the present invention.

FIG. 18 is a diagram showing an embodiment of a code conversion block and a character type determination block according to the second embodiment of the present invention.

FIG. 19 is a diagram showing one embodiment of a 2-byte character code conversion circuit according to the second embodiment of the present invention.

FIG. 20 is a diagram showing one embodiment of a 1-byte character code conversion circuit according to the second embodiment of the present invention.

FIG. 21 is a diagram showing one embodiment of a simplified character code conversion block according to the second embodiment of the present invention.

FIG. 22 is a diagram showing an example of a conversion format from the SJIS code system to the incompletely compressed character code system.

FIG. 23 is a diagram showing one embodiment of a 2-byte character code conversion circuit according to the third embodiment of the present invention.

FIG. 24 is a diagram showing one embodiment of a 2-byte character code conversion circuit according to the fourth embodiment of the present invention.

FIG. 25 is a diagram showing one embodiment of a simplified 2-byte character code conversion circuit according to the fourth embodiment of the present invention.

FIG. 26 is a diagram showing an example of a conversion format from an EBCDIK / KEIS code system to a compressed character code system.

FIG. 27 is a diagram showing one embodiment of a code conversion block and a character type determination block according to the fifth embodiment of the present invention.

FIG. 28 is a diagram showing an embodiment of the escape sequence detection circuit according to the fifth embodiment of the present invention.

FIG. 29 is a diagram showing an example of a character type determination table for 1-byte characters according to the fifth example of the present invention.

FIG. 30 is a diagram showing an example of a character type determination table for double-byte characters according to the fifth example of the present invention.

FIG. 31 is a diagram showing an example of a conversion format from the JIS / JIS code system to the compressed character code system.

FIG. 32 is a diagram showing an example of the code converting means according to the sixth example of the present invention.

FIG. 33 is a diagram showing an example of a bit width conversion means according to a sixth example of the present invention.

FIG. 34 is a diagram showing an example of a character code acquisition window according to the sixth example of the present invention.

FIG. 35 is a diagram showing one embodiment of a character code selection means according to the sixth embodiment of the present invention.

FIG. 36 is a diagram showing an embodiment of a character code selection means according to the sixth embodiment of the present invention.

FIG. 37 is a diagram showing the operation of one embodiment of the code converting means according to the sixth embodiment of the present invention.

FIG. 38 is a diagram showing one embodiment of the escape sequence detection circuit according to the sixth embodiment of the present invention.

FIG. 39 is a diagram showing values in a character code conversion table.

FIG. 40 is a diagram showing values in a character code conversion table when a different notation search is not performed.

FIG. 41 is a diagram showing values in a character code conversion table when a different notation search between half-width and full-width characters is performed.

FIG. 42 is a diagram showing values in a character code conversion table when performing a different notation search between half-width and full-width characters in katakana.

FIG. 43 is a diagram showing values in a 1-byte character code conversion table.

FIG. 44 is a diagram showing values in a code conversion offset table.

FIG. 45 is a diagram showing values in a code conversion offset table.

FIG. 46 is a diagram showing values in a code conversion offset table in the third embodiment.

FIG. 47 is a diagram showing values of a 1-byte character code conversion table in the fifth embodiment.

FIG. 48 is a diagram showing values in a 2-byte character code conversion table in the fifth embodiment.

FIG. 49 is a diagram showing values in the code conversion table for 1-byte characters in the sixth embodiment.

FIG. 50 is a diagram showing values in a 2-byte character code conversion table in the sixth embodiment.

[Explanation of symbols]

1 ... Character string search device, 105 ... Character string storage means, 400
... character code conversion means, 102 ... character string collation means, 41
0a, 410b ... Character code acquisition means (window), 420 ... Character code selection means, 430 ... Code conversion means

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Atsushi Hatakeyama 1-280, Higashi Koigokubo, Kokubunji, Tokyo Inside Central Research Laboratory, Hitachi, Ltd. (72) Inventor Masatsugu Shinozaki 810 Shimoimaizumi, Ebina, Kanagawa Prefecture Hitachi, Ltd. Office system division

Claims (23)

[Claims] 1. A character code conversion device used for exchanging text data between devices having different character code systems, comprising a plurality of character code capturing means for capturing input character codes in a state where they are shifted by 1 byte from each other. A character code selecting means for selecting a character code to be output from the character codes output by the plurality of character code fetching means, based on the immediately preceding selection result and the code conversion result, and the character code selecting means. A code conversion means for discriminating a character type such as whether the generated character code is a 1-byte character or a 2-byte character, and performing a character code conversion corresponding to each character type. The input text written in one character code system is changed to the second character code different from the first character code system. Character code conversion apparatus characterized by converting the de scheme. 2. As the code conversion means in the character code conversion device according to claim 1, when the code conversion is performed on the input text written in the first character code system, the code is allocated without a gap. A character code conversion device having a complete compression type code conversion means for compressing and converting to a second character code system. 3. As the code conversion means in the character code conversion device according to claim 2, the input text written in the first character code system is compressed into a second character code system in which codes are allocated without any gaps. A character code conversion device having a complete compression type code conversion means for converting and a character code reverse conversion means for converting a complete compression type code system back to a non-compression type character code system. 4. A code conversion means in the character code conversion device according to claim 1, wherein when performing code conversion on an input text written in the first character code system, a code is allowed with a gap partially. A character code conversion device having an incompletely compressed code conversion means for compressing and converting into a second character code system allocated to. 5. A second character code as code conversion means in the character code conversion device according to claim 4, wherein the input text written in the first character code system is assigned a code with a gap partially allowed. A character code conversion device having not-completely-compressed code conversion means for compressing conversion into a system and character-code reverse conversion means for performing reverse conversion from an incompletely-compressed code system to a non-compressed character code system . 6. A code conversion unit in the character code conversion apparatus according to claim 1, which has a code conversion circuit for a 1-byte character and a code conversion circuit for a 2-byte character independently, and a code according to a corresponding conversion mode. A character code conversion device comprising a conversion mode selection type code conversion means for converting a character code by selecting a conversion value. 7. A code conversion means in the character code conversion device according to claim 1, which is a batch processing type code conversion means for performing code conversion processing for 1-byte characters and code conversion processing for 2-byte characters in the same circuit. A character code conversion device having. 8. Character type information for determining whether the character code selected by the character code selecting means is a 1-byte character or a 2-byte character as the code converting means in the character code converting device according to claim 1. A character code conversion device having a character type information storage means for storing at least one or more types, and performing character code conversion according to the character type information obtained by the character type information storage means. 9. The character code conversion device according to claim 8, wherein in addition to the character type information such as whether the input character code is a 1-byte character or a 2-byte character, the input character code is stored in the character type information storage means. Character code conversion characterized in that conversion processing information necessary for conversion into different character code systems of different types is stored, and character code conversion is performed using the character type information and the conversion processing information obtained by the character type information storage means. apparatus. 10. The character code conversion device according to claim 8, wherein when converting an input character code into another different character code system, for input text in which character strings represented by a plurality of code systems are mixed. A character code conversion device characterized by dynamically converting a code. 11. A document retrieval apparatus for retrieving a document containing a retrieval word specified in a retrieval condition expression for a document database stored as character codes, wherein input character codes are fetched in a state of being shifted by 1 byte from each other. A plurality of character code fetching means, and a character code selecting means for selecting a character code to be output from the character codes output by the plurality of character code fetching means based on the immediately preceding selection result and the code conversion result; The character code selecting unit has a code converting unit that determines a character type such as whether the character code selected is a 1-byte character or a 2-byte character, and performs character code conversion corresponding to each character type. The input text written in the first character code system in which double-byte characters are mixed is used as the first character code. Document search apparatus characterized by performing the matching process for the specified search term after converting systematically different from the second character coding system. 12. The code conversion means in the document retrieval device according to claim 11, wherein when the code conversion is performed on the input text written in the first character code system, the code is allocated without any gap. A document retrieval device comprising a complete compression type code conversion means for performing compression conversion to a second character code system. 13. The code conversion means in the document retrieval device according to claim 11, wherein when the code conversion is performed on the input text written in the first character code system, the code is allowed with a gap partially. A document retrieval device having an incompletely compressed code conversion means for compressing and converting into the allocated second character code system. 14. A code conversion unit for a document retrieval apparatus according to claim 11, which has a code conversion circuit for a 1-byte character and a code conversion circuit for a 2-byte character independently, and code conversion in a corresponding conversion mode. A document retrieval apparatus having a conversion mode selection type code conversion means for converting a character code by selecting a value. 15. The batch conversion type code conversion means for performing code conversion processing for 1-byte characters and code conversion processing for 2-byte characters in the same circuit as code conversion means in the document retrieval apparatus according to claim 11. A document retrieval device characterized by the above. 16. The character type information for determining whether the character code selected by the character code selecting means is a one-byte character or a two-byte character as code conversion means in the document search device according to claim 11. A document search characterized by having character type information storage means for storing at least one kind or more, and performing collation processing of a specified search word after performing character code conversion according to the character type information obtained by the character type information storage means apparatus. 17. The document retrieval apparatus according to claim 16, wherein the input character code is stored in the character type information storage means in addition to the character type information such as whether the input character code is a 1-byte character or a 2-byte character. A code conversion information storage type code conversion means for storing conversion processing information necessary for conversion into a different character code system and performing character code conversion using the character type information and the conversion processing information obtained by the character type information storage means. A document search device having: 18. The document retrieval apparatus according to claim 16, which is unnecessary for performing retrieval in addition to character type information such as whether the input character code is a 1-byte character or a 2-byte character in the character type information storage means. In addition to storing specific character type information corresponding to a control code, that is, a specific control code representing double-width display, halftone display, etc., in addition to performing code conversion according to each character type information as code conversion means,
A document retrieval device comprising control code deletion type code conversion means for deleting the character code when the character type information obtained by the character type information storage means represents the specific character type. 19. The document retrieval device according to claim 16, wherein in addition to character type information such as whether the input character code is a 1-byte character or a 2-byte character in the character type information storage means, a document number, a page number, and an indent. In addition to storing specific character type information corresponding to a specific control code added before the binary data such as quantity, and performing code conversion according to each character type information as code conversion means, A document retrieval apparatus comprising a binary data detection type code conversion means for detecting binary data when the obtained character type information represents the specific character type. 20. The document retrieval apparatus according to claim 16, wherein the character type information storage means stores the code conversion information only for the character code included in the retrieval word specified in the retrieval condition expression, and the other characters. Specific character type information is stored for the code, and when the character type information obtained by the character type information storage means as the code conversion means represents the specific character type, it is deleted as a character code not related to the search word. A document retrieval device having unnecessary code deletion type code conversion means. 21. In the document retrieval apparatus according to claim 16, in addition to character type information such as whether the input character code is a 1-byte character or a 2-byte character in the character type information storage means, the end of the file to be searched, etc. A specific character type information corresponding to a specific control code having a special meaning is stored, and a document search process is performed when the character type information obtained by the character type information storage means as the code conversion means represents the specific character type. A document retrieval apparatus having a control information detection type code conversion means for outputting control information such as the end of. 22. In the document retrieval device according to claim 16, various characters are included in the input text due to different notation expressions such as half-width characters such as alphanumeric characters, uppercase and lowercase letters, Katakana Hiragana characters, old and new Kanji characters. When converting a character string represented by a code to another character code system different from the input character code system, by converting the code to a certain character code string, A document retrieval device characterized in that the retrieval processing is performed collectively. 23. When converting an input character code into another different character code system in the document retrieval apparatus according to claim 16, a dynamic operation is performed for input text in which character strings represented by a plurality of code systems are mixed. A document retrieval device having a code conversion means for converting a code into.