SU1418757A1 - Device for lexical analysis of programs - Google Patents

Abstract

The invention relates to computing technology, is intended for hardware support of the translation process, and can be used in specialized processors or software and hardware that carry out the translation from high-level programming languages to computer-based programming, as well as in high-level programming language interpreters. The purpose of the invention is the extension of functionality due to the provision of lexical analysis in a free format and recognition of a token. A character string. , a 12 character decoder and a control unit 13 are additionally introduced. 4 yut., 1 tab. S

Description

00

 &one

VI

ttf

one . U

The invention relates to computing technology, it is intended for I hardware support of translation processes and can be used in specialized processors or software and hardware means that perform translation from high-level programming languages to machine languages. , as well as in the interpreters of programming languages: high level, i. The purpose of the invention is to expand the functionality by providing lexical analysis, programs of high-level IB languages, free format and recognition

; Character string,. . The essence of the invention is to create a device for lexical program analysis that is capable of producing typical programming lexemes of high-level programming in a free format using the symbol of the program and decoding with it the class of characters and identifying the control unit on the basis of this class of lexical units.

Fig. 1 shows the functionality of the device diagram for lexical analysis of programs; in fig. 2 is a functional diagram of an example implementation of a control unit; FIG. 3 - firmware § in FIG. 4 - time diagram of the operation of the device.

The device for lexical analysis of the program contains a conversion unit 1; register 2

groups 4 and 5 elements AND, inputs 6, information 7, start 85 information output 9, outputs 10 and readiness llj decryptor 12 characters, control block 13,.

There is a correspondence between the constructions of lexical units and x types and values given by the compiler's developers, “The value of a lexeme can be either a sequence of 5-character symbols — this construct, or some code. The value field for some lexemes may be absent. The unit 1 converts the code of the Liter into the Lexey code and sets the correspondence between the leximi- cvsk-tt structures and their types. At the same time, the block 1 also sets the values T

,

.one

code of the letter in the code of Toksem .5 group 3. elements OR,

five

0

five

five

O 0

0

lexeme, if represented by code. The code can .present values of tokens: keywords, separators. Values for tokens. A character string, an identifier and a number are formed from a sequence of characters forming them,

Register 2 is assigned to receive and store up to-letter input program text and can be assembled on parallel or series action shift registers. The next letter is entered from the information input of the register 2 into the junior ity of the register with the simultaneous shift co. held by register 2 from the lower order bits to the OLDER, but the number of bits is equal to the number of bits in the character code in the case of a pulse input to the shift input of the register 2.

Groups 4 and 5 elements. And are intended for switching, respectively, tokens- (type) and the corresponding letter (value) through group 3 elements OR to information output 9 of the device:.

A decoder of 1-2 characters is intended for generating a control code using character codes located in the most significant bits of register 2. The control code: corresponds to the length of these symbols. The following classes are highlighted; character in high-order bits of register 2, letter | digit character; a character that restricts a character string (for example, a quotation mark); comment start character j comment end character; the character that is the one lame delimiter; the symbol and the one following it, which are two-sided: Separator; the character that is the highlighted end-of-program character; a sequence of characters in register 2, starting with a character in the higher bits, forms a key word.

The control unit 13 analyzes the incoming input (awareness) signals and the generation of output (control) signals by which the device operates in accordance with the lexemes recognition algorithm.

The device works as follows.

At the initial time, register 2 is zero. The device is started by applying a pulse to the input 8, on which the block 13 starts working

control, it sends a pulse to the output 10 of the device, along which the first character of the program must be fed to the input 7 of the device. After the input of the code 7 of the simnol to the input 6 of the device, a pulse is applied confirming the presence of the character at the input 7. After this, the control unit 13 sends a pulse to the output 10, and the symbol reading cycle repeats until the entire register 2 is filled. the latter is determined by the generation at the output of the decoder of 12 characters of code, distinct from zero. After register 2 is filled in, block 13 begins to recognize the tokens of the program, depending on the class of characters received at the input of the decoder 12 characters.

If the control code corresponds to the key M characters, this means that in register 2, beginning with the highest bits, the characters of the keyword are found. In this case, the type output (and, if necessary, value) of the token Keyword is generated at the information output of block 1. A logical unit is supplied to the second input of a group of 4 elements AND from block 13, and the generated type of token is transmitted through a group of 3 elements OR to information output 9 of the device. At the same time, the output 11 of the device is given a pulse signaling readiness at the information output 9 of the token device. Thereafter, the characters in register 2 are shifted from the least significant bits to the most significant ones, while the lower ones of the new symbol of the source program are received from the information input of the device. The shift is carried out before all characters of the given keyword are ejected from register 2. The end of a keyword is determined by the appearance of 2 characters other than a letter in the upper bits of the register by analyzing the control code at the output of the decoder of 12 characters.

Simultaneous reception of new program symbols is carried out by sending a pulse at the output 10 of a request for a symbol and waiting for the appearance of a single signal at input 6. After a pulse arrives from input 6, indicating the presence of the current program symbol at input 7

15

20

25

418757

the shift of characters is in register 2 with the reception of a new bit in the lower bits. Further, the reception of program symbols is carried out in the same manner. After forming the keyword token and shifting its characters in register 2, the polling of the control code at the output of the decoder of 12 characters is resumed and the actions determined by the class of 12 characters analyzed by the decoder are executed. .

If the character in the high bits of register 2 is a letter, this indicates the beginning of the identifier. At the information output of block 1, a type of lexeme is formed, which is transmitted through a group of 4 elements AND and a group of 3 elements of the RShI to the information output 9 of the device by sending a corresponding signal from the control unit 13. At the same time, an output is supplied to the ready output 11: a pulse, indicating the presence of a lexeme-type device on-output 9. After that, the identifier symbol is transmitted to device ID 9 through a group of 5 elements AND and a group of 3 elements OR by submitting 5 elements AND to the second input of a group. At the same time, an output is sent to the output 11 of readiness implying the transfer of an identifying letter to the information output 9 of the device (i.e., the value of this lexeme). devices in this way, a new program character arrives. All identifier characters are transmitted to information output device 9 in a similar manner. . .

The end of the characters of the identifier is determined by the appearance of 2 characters in the upper bits of the register, different from the letter and number. In this case, a zero code is transmitted to the information output 9 of the device, indicating the completion of the sequence of identifier symbols. This is done by sending a pulse to the output 11 of the readiness of the device, since logical units are not supplied to the second inputs of groups 4 and 5 of the elements AND from the control unit 13, there are zero codes at their outputs which, through the group

thirty

35

40

45

50

55

3 OR elements are transmitted to information output 9 of the device. After

I transmissions of a null code are performed in accordance with the class of c symbols, which are in the high bits of register 2,

If the character in the high bits of register 2 is a digit, it means the beginning of an integer. At the information output unit 1 is formed

i is a type of token, which is transmitted through a group of 4 AND elements and a group of 3 OR elements to the information output 9 of the device by submitting a corresponding signal from 15 control unit 13. At the same time, output 11, a readiness, is given a pulse, indicating the presence of a lexeme-type device at output 9. After that, 20 at the output of the device 9, the digits of the number are transmitted through a group of 5 elements AND and a group of 3 elements OR by sending a corresponding signal from block 13 .-- At this, exit 11 is ready-25 to carry a pulse indicating the transfer to information

The output is 9 digits (i.e., the meaning of this token). Simultaneously with in-. of the formation input 7, the new symbol of the program enters in this way. Similarly, all digits of a number are transmitted to the information output of the device. End digits of a number is determined by the appearance of a character other than digit in the upper 35 bits of the .2 register. In this case, a zero code is transmitted to the information output 9 of the device in this way indicating the completion of the 40 digit sequence of numbers. After transmitting the zero code, actions are performed that correspond to the class

. characters in the higher bits of the register 2.45

If the character in the high bits of register 2 is a one-character separator and does not form a two-type separator with the next after it, on the information block of 50

The lexeme type is formed. The one-letter separator (and, if necessary, its value). This type is transmitted through a group of 4 elements AND and a group of 3 elements S1I to the information output 55 of the device 9 by supplying a corresponding signal from the control unit 13. Simultaneously at the output

A second pulse is given, swi.

detentions about the presence at output 9 of the device is a type of lexeme. After that, the contents of register 2 are shifted with the entry of the original program to the lower bits of the new symbol. Further actions of the device are determined depending on the class of symbols located in the high bits of the register 2.

If the character in the high bits of register 2 and the next after it are two character delimiters, the token type is formed on the information output of block 1. A double type separator (and, if necessary, its value). In this case, the device acts all are similar to the actions performed during the formation of one-liter separators, but in the end, the register 2 is shifted by two characters by two characters with the reception of new program characters.

If the character in the high-order bits of register 2 is a sign of a character string (for example, a quotation mark), the type of the token type is formed at the information output of block 1. A character string. This type is transmitted to the information output 9 of the device in the same manner as for the rest of the tokens. After the transfer of the type, the contents of register 2 are shifted by one character from the received new program character. Since a character string represents from itself a sequence of characters, bounded on both sides by special features (for example, quotation marks), the value of this lexeme (the characters of the character string itself) is transmitted until the appearance of the second character of the character string in the upper 2 bits of register 2. All characters of the character string, which represent the value of the token, are transferred from register 2 to information output 9 of the device, as well as the symbol (token value) of the identifier. The terminating character of the character string is pushed out of register 2 (with the reception of a new character), and the device continues to work depending on the class of characters located in the high order bits of register 2.

If the character in the high bits of register 2 is the character of the beginning of a comment, block 13 shifts the contents of register 2 (with the reception of new characters) until it appears in the high bits of the last character End of the comment, which also shifts - ,with . After that, the device operates depending on

classes of characters that are in the high bits of the register 2.

If the character in the high bits of register 2 is the end of program character, the device

c character codes that are in register 2. They denote each bit of register 2 as a Boolean variable, where P is the number of the bit. Building a system of Boolean functions T begins with building systems of Boolean functions for each of the different lexemes. For keywords, a system of Boolean functions that form a type of keywords is defined, for K, for identifiers through I, for L1 separators, for K1, for double digits - through R2,

completes its work. Since the word-15 for a number is N, for character

The device operates in the request-response mode and, after requesting the next character, waits for its arrival using the Response input polling, after transmitting the last program character (end character), it is necessary to transmit a pulse to the response 6 input several times (without actually setting the character code on information input 7) ) until the end of the program, the end of program symbol will move from the lower bits of register 2 to the higher ones.

If the character in the high bits of register 2 does not belong to any of the listed groups of characters (the non-positive code at the output of block 12), it is assumed that this character belongs to a group of characters of the type Space, Tabulation, Line break and etc. The characters from this group must be skipped, which is accomplished by shifting the contents of register 2 and receiving a new character. Then, the actions of the device are resumed depending on the class of characters in the higher bits of the register 2.

Building a system of boolean functions that implement block 1 is as follows.

The initial data for building block 1 is the correspondence between the structures of lexical units and their types and meanings (they assume that the types and meanings in it are represented as binary numbers). Block 1 defines the correspondence between the lexical constructions / and their types, and it also sets the value if it is represented by a code. Two systems of Boolean functions are considered: the T-defining type of the lexeme and the M-defining value of the lexeme. Argu: 1tams are 5

C -, - n

, constants - through S, Consider the construction of each of the listed systems separately.

0 Zero functions f- are defined as follows.

0- if in the bits of register 2 in place, the i-ro symbol of the ox (starting with the higher bits of register 2) is a symbol or 1difra).

1- otherwise

 These functions are easy to implement, for example, by simply enumerating elementary conjunctive ears in the disjunctive normal form (DNF), which take the value of a unit on sets of values corresponding to hg. Codes of 5 oxen, different from letters and numbers,

In the future, use these functions.

The system of Boolean functions can be represented as

40

.I;

k,., (l;:, l;, r1.,).,;

K, v, (A ,, zVt) & 4 ui;

K, vt (4H ;, zt-,) f. ,,,

p where each Z: {. (,,. .h "; j 1, ..., m;

, 8 5; , ..., K) there is either a variable y or the negation of the variable Ur f the number of keywords for which in the discharge of a type code is the number of letters

there is a unit; g

in the 1-M keyword, in which in the 1 st category of the type code is one; 55 m is the number of bits needed to encode source symbols; K is the number of bits needed to encode the types of tokens.

The character after the keyword must not be a letter or a number, otherwise it is not a keyword, but an identifier. Therefore, each elementary I.., V i. W f

: in conjunction of the species L., A, 7,.,

:. . Tg T) 1 1J t

takes the value of one on the set of variable values, which corresponds to a sequence of g. | letter codes for a certain keyword, and generally a conjunction

vanish by f „f

: ir;

1 + -1

1- otherwise.

Functions q W are trivially implemented, because for each i-ro one; In case after the last letter

keyword followed by a letter or

 Numeral. Obviously, recognition

keywords, like other tokens,

carried out for those of them, the beginning of the 20 letter separator characters, together: which is located in the register with which it forms a two-interleaver 2 with the most senior bits. the new separator is a finite number

I Since the identifier is obsessive- (usually no more). When,

| but must begin with a letter and one if each of these characters is represented

the letter is also an identifier, 25 corresponding to an elementary conjunk type for an identifier with the help of the i-system of the boolean functions I can be formed according to the character codes located in the high-order bits of register 2.

In DNF and take the negative from the resulting DNF, the corresponding q is obtained. V

R1 system

1 VrV. (Aj.z | j) q ;;:

. g

m

, 2

for separators from

| For the digits of the type code, the identifier-° of the second group is of the form ra, which is equal to one, all functions I have the form

1g, zj. ,

p S where each Z j (, ..., 6; j 1, ... m |

, ... K) is either a variable y, or a negation of the variable Ur, and these are the corresponding variables

35

Rl2 Vj ,, (Aj ,,) R4 Vb,. () “Q,;,

/

where each Z- (, ..., r; j 1, ..., m;, ..., K) is either a variable Ur.

.... . - .. “.

the senior bits of register 2, in which 40 is the negation of the variable y., and

The roar is the first character of a recognizable construct; b - the number of letters in the alphabet language; each elementary conjunction Ajl, Z - takes the value of one on the set of values of variables corresponding to some code of the letter.

The remaining Ig functions are identically zero.

One-liter separators can be divided into two groups; the first is a group of delimiters, which cannot be the beginning of two-delimit delimiters, and the second is delimiters

these are the variables corresponding to the most significant bits of register 2, in which the first character of the recognizable structure is found; g g is the number of dividers, for which there is one in the 1st category of the type code.

Kalada elementary conjunction dt, /

 J- takes the value of the unit on

a set of values of variables that gg correspond to a single-letter separator, but this conjunction generally vanishes by the corresponding function q. in case the first and second characters of the register

which can be the beginning of a two-gg 2 form a two-splitter. S) 1X separators. For one-liter systems, the system of Boolean functions R2 for separators of the first group of the system of two-liter separators builds three functions R1 is constructed in a trivial, abundant manner, in this case at a time .. the corresponding DNF elements

Consider constructing R2 for the second group / single line separators. For this, all the one-liter dividers from the second group are renumbered and for each of them a function is entered (j0 - if in the bits of register 2 the q character is replaced by the second character (starting with the higher bits); J forms the symbol that forms i m single-character separator is a two-character separator .;

1- otherwise.

The functions q W are trivially implemented, because for each i-ro by single section in DNF and taking the negative from the obtained DNF, the corresponding q is obtained. V

R1 system

for separators from

1 VrV. (Aj.z | j) q ;;:

the group swarm looks like

. g

m

, 2

Rl2 Vj ,, (Aj ,,) R4 Vb,. () “Q,;,

/

where each Z- (, ..., r; j 1, ..., m;, ..., K) is either a variable Ur.

.... . - .. “.

 denial of variable u., and

these are the variables corresponding to the most significant bits of register 2, in which the first character of the recognizable structure is found; g g is the number of dividers for which there is one in the 1st category of the type code.

Kalada elementary conjunction dt, /

 J- takes the value of the unit on

a set of variable values that correspond to some one-liter separator, but this conjunction generally vanishes by the corresponding function q. in case the first and second characters of the register

111418757

conjunctions correspond to codes of two stegols, forming a two-double separator.

The system of Boolean functions S for a number is constructed like a system of functions for an identifier except for B

dc ec ca bu

T -

T K; v () V.R1 Tr (() VR1, JvR2jW2VS2

T K, V (lf (I,) (,

letters and numbers.

The system of Boolean functions S for a character string consists of functions that are identically equal to zero if they correspond to zero digits in. the code type of the character string, and functions consisting of one elementary conjunction corresponding to the character code of the character string of the character string (for example, quotation marks), the functions correspond to one bits in the code type of the character string.

Build a system of Boolean functions T, which specifies the type of tokens in register 2, it is a superposition of these systems of Boolean functions.

where is tf

The If function takes the value of one, if in register 2, starting with the higher bits, the key word is located, and zero otherwise. This function forms a keyword tag that is generated at the corresponding output of block 1.

Building a system of Boolean functions M to form lexeme values that are specified by code is, in all, similar to building a system (1) of functions T.

Consider the construction of a system of Boolean functions that implement the decoder 12 characters. A 12 character decoder is considered to produce a unitary control code. Then each output of the decoder 12 characters corresponds to a separate Boolean function, which for the corresponding group of characters located

in the higher bits of register 2, received the code of the ROM 14 is connected to the input, the value of the unit, and for the rest of the register 15 micro-commands. In accordance with .

They consider each function separately.

with the micro-command structure, the outputs of the register of the 15 micro-commands are connected: the bits of the field Y are equal to

12

The function in the character group for a letter is

In v

BUT;;.

ve

where each (,.,., b;,.,., m) is either a variable y, or the negation of the variable Ur b is the number of letters in the alphabet.

Ka da elementary conjunction

corresponds to some book

Jm

ve

Function c for a group of characters. A digit is constructed in a similar way.

The functions H and Hj of the beginning and end of comments are constructed in the same way as the function W of the character string feature.

The functions F and FJ of one-sided and two-sided separators are constructed respectively as superpositions of functions R1 and R2j.

v,., Rii;

Fo vr. R2;

The function P of the attribute key has the form

lay

P vj to ..

Obviously, that the functions that are implemented by blocks 1 and 12 allow minimization, which reduces hardware costs when implementing these blocks.

The control unit 13 can be implemented, for example, in the form of a firmware automaton using a ROM (FIG. 2). It uses forced addressing and two-address fields. A Kalda microinstruction has an operational field Y, a field of logical conditions X and two address fields A0 and A1. Microprogrammed control automaton contains ROM 14, micro-command register 15, decoder 16, AND-OR element 17. 1K-trigger 18, HE element 19, AND element 20, multiplexer 21, generator 22 sync pulses, first through five outputs 23 the automaton, from the first to the eleventh inputs of the 24 automaton, the input 25 of the launch of the automaton.

 Vcode ROM 14 is connected to the input of the register 15 microinstructions. According to

with the micro-command structure, the outputs of the micro-command register 15 are connected: the bits of the field Y are the corresponding

23 automaton outputs, bits of field X — with the input of the decoder 16, bits of the fields A0 and A1 — with the address Input of the ROM 14, controlled through a mule typelexer 21 respectively with the outputs of the AND-OR element 17 and the HE element 19, the decoder outputs 16 and inputs 24 of the automaton - with the inputs of the element AND-OR 17, the first bit of the floor Y) of the register 15 is connected to the input 1K of the trigger 18, to the 1-input of which receives a start signal, and the direct output of which is connected to the first input the element And 20. At the second input of the last received clock pulses from the generator 22 clock pulses. The output of the element And 20 is connected to the input of the reading of the ROM 14.

Firmware control auto Tsat works as follows,; In the initial state, the rgist 15 microinstructions and trigger er.18 are located. in the zero state, as a result of which the zero address of the ROM is set. A start pulse from input 25 switches the 1K-trigger 18 to a single state, as a result of which the clock pulses pass to the read input of ROM 14. Exciting in each step the microcommand reading process from ROM 14, the microcommand selected from ROM 14 enters the register 15 microinstructions and processing are as follows. Field Y forms control signals. The address of the next microcommand is defined by memory A0 and A1 depending on the value of field X and logical conditions received by the input group 25 "If the condition selected by the decoder 16 is zero or and in the X field the condition is identically equal to zero (a microcommand without checking the logical condition), is selected by the cocking of the element NOT 19 is address A1, and if the condition specified in the command is equal to one, by the output of the element AND-OR 17 the address A is chosen as addresses of the next microinstruction. The operation of the microprocessor control automaton is terminated if a specific stop microcommand is specified. This microinstruction has a unit in the first digit of the Y register of 15 and zeros in the remaining bits. According to this micro-command, the first bit of register 15 receives a signal at K input of a 1K-flip-flop 18, bringing it to the zero state and

the most prohibiting the passage of clock pulses through the element And 20,

In the event that a microprogram control automaton is selected as the automaton control automaton, the operation of the device is determined by the microprogram for this automaton. When describing a microprogram, the language Fc of the national microprogramming is used (F-language), which is a means of describing the functions of operating devices at the microprogram level irrespective of the structure (hardware) that can be used to implement these functions. Fz is a microprogram that contains two kinds of information, two parts; description of words and arrays, establishing the types and formats of words with which the firmware operates; content graph firmware; which defines the algorithm for executing operations in a meaningful form, in the form of descriptions of microoperations and logical conditions.

The set of words the microprogramme works with is described in the form of a table.

It is assumed that the start of the control unit 13 is carried out before the start of the microprogram operation by applying a pulse to the input 8 of the device.

The content graph of the firmware t-fbi corresponding to the algorithm of the device operation is shown in FIG.

FIG. 4 shows a time diagram of the operation of the device during the implementation of a lexical analysis of a program;

Claims (1)

Invention Formula A device for lexical analysis of programs containing a block for converting a code of a letter into a code of a token, a register, a group of OR elements and two groups of AND elements, the information input of the register being the information input of the device, the information outputs of the register corresponding to individual characters , are connected to the information inputs of the conversion block of the code of the Letter to the code of LeXem, the inputs of which are connected to the first inputs of the AND elements of the first group, the outputs of which are connected to the first inputs of the OR elements, the outputs of which x data outputs form the device - : .15 . 1418757 the second inputs of the elements OR of the pa group are connected to the outputs of the elements AND of the second group, the first inputs of which are connected to the outputs of the register corresponding to the first character entered in the non-, r6 character, characterized in that, in order to extend the functionality by provision of lexical analysis of high-level language programs in a free Format and recognition of a token A character string, into it the character decoder and the control unit are entered, the first and second information inputs of the character decoder are connected to the outputs of the register 10 15 rum first feather from mouth of mouth 57 ra, sixteen corresponding to the first and second characters entered into it, the output of the character decoder Connected to the first input of the control unit, the second and third inputs of which are the start and tracking inputs of the device, respectively, the first and second outputs of the control unit pod-. Are connected to the second input s of the elements AND of the first and second groups, respectively, the third output of the control unit is connected to the synchronization input. the register, the fourth and fifth output of the control unit are readiness and request outputs of the device, respectively. . 2J T Y 18 25 about- go FROM g nineteen & tO-1 2 five G. 1 4 5 "about him i fl 1- a «t