KR20230017935A - Chemistry education device using artificial intelligence and augmented reality and method of providing chemistry education service using the same - Google Patents

Abstract

A chemistry education device using artificial intelligence and augmented reality and a providing method of a chemistry education service using the same are provided. The chemistry education service includes an AI tutoring service and an AR implementation service. In particular, the AI tutoring service recognizes chemical symbols, numbers and other symbols that make up chemical reaction formulas, extracts the context of the chemical reaction formulas by using a positional relationship between them, extracts an error by comparing the chemical reaction formulas with a reference reaction formula, and visualizes a correct answer by augmented reality to provide the same.

Description

Chemistry education device using artificial intelligence and augmented reality and method of providing chemistry education service using the same

The present invention relates to a chemistry education device and a method of providing chemistry education services using the same, and more particularly, to a chemistry education device using artificial intelligence and augmented reality and a method of providing chemistry education services using the same.

As a result of a survey of Bangladeshi school students, it was found that they had the most difficulty in understanding and visualizing chemical formulas, chemical names, and chemical equations in school education. These difficulties cause many mistakes in school tests and act as a cause of lowering the desire for academic achievement. Accordingly, the inventors of the present invention have completed the present invention so that difficult chemical terms or chemical reactions can be easily understood by visualizing difficult chemical terms or chemical reactions using new technologies of augmented reality and artificial intelligence to solve the difficulties of students.

The problem to be solved by the present invention can be visualized and displayed in augmented reality to easily understand difficult chemical terms, and if there is an error in the chemical reaction formula, it can be automatically extracted and the correct answer and its chemical structure can be displayed in augmented reality. It is intended to provide a method of providing chemistry education services that are available.

Another problem to be solved by the present invention is to provide a chemistry education device that provides such a chemistry education service.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood by those skilled in the art from the description below. will be.

A method of providing chemistry education services according to an embodiment of the present invention for achieving the above object is a method of providing chemistry education services using a chemistry education device including a camera unit, an output unit and a database, wherein the chemistry education device Optionally, it may include providing an AI tutoring service or an AR implementation service. Here, the providing of the AI tutoring service may include photographing the object and displaying image information of the object on the output unit; detecting a chemical reaction formula described on the subject; Recognizing chemical symbols, numbers and other symbols constituting the chemical reaction formula; extracting the context of the chemical reaction formula by using the positional relationship between the chemical symbol, the number, and the other symbols; Defining a plurality of reference reaction formulas in advance in the database, and extracting errors by comparing the chemical reaction formula with the reference reaction formula; If there is an error in the chemical reaction equation, displaying a bounding box around the error on the output unit; and displaying a correct answer to the error on the output unit, visualizing a chemical structural formula in the correct answer in augmented reality according to a user's selection, and displaying it on the output unit.

The step of extracting the context may include classifying one side as a reactant and the other side as a product based on an arrow or an equal sign in the chemical reaction equation; And in the positional relationship between the chemical symbol and the number, (i) when the number is arranged in front of the chemical symbol, (ii) when the number is arranged in a subscript after the chemical symbol, and ( iii) extracting the context of the chemical reaction formula by identifying a case in which the number is placed in a superscript at the rear of the chemical symbol.

In the step of extracting the error, a reactant made of one or more materials and a product made of one or more materials are distinguished from the chemical reaction formula, and a reference reaction formula having the highest degree of agreement based on the materials included in the chemical reaction formula is obtained from the database. choosing; extracting formula errors by comparing chemical formulas of reactants and chemical formulas of products for the chemical reaction formula with chemical formulas of reactants and chemical formulas of products for the reference reaction formula, respectively; and extracting coefficient errors by comparing coefficients of reactants and coefficients of products for the chemical reaction equation with coefficients of reactants and products for the reference reaction equation, respectively, after correcting the chemical formula error.

The providing of the AR implementation service may include photographing an object and displaying it on the output unit; detecting a chemical term written on the subject (the chemical term includes a chemical name and chemical formula); predefining a plurality of reference terms in the database, and displaying a bounding box around the chemical terms on the output unit when the chemical terms coincide with the reference terms; When a selection event occurs for the bounding box, a selection is selected from the group consisting of the 'Reading method' option window, the 'View molecular formula' option window, the 'View structural formula' option window, and the 'View additional information' option window for the chemical term. displaying at least one option window on the output unit; and displaying educational information corresponding to the selected option window on the output unit when a selection event occurs for any one of the option windows.

A chemistry education device according to an embodiment of the present invention for achieving the other object is a chemistry education device providing a chemistry education service, comprising: a camera unit for photographing an object; an output unit displaying an image of the object; an AI tutoring unit providing an AI tutoring service for the chemical reaction described on the subject; an AR implementation unit providing an AR implementation service for chemical terms written on the object; And a plurality of reference reaction equations and a plurality of reference terms may be included in the stored database defining in advance. Here, the AI tutoring unit detects a chemical reaction formula described in the object from the image information displayed on the output unit; Recognize chemical symbols, numbers and other symbols constituting the chemical reaction formula; Extracting the context of the chemical reaction equation using positional relationships between the chemical symbol, the number, and the other symbols; extracting errors by comparing the chemical reaction equation with the reference reaction equation; if there is an error in the chemical reaction equation, display a bounding box around the error on the output unit; A correct answer to the error may be displayed on the output unit, and a chemical structural formula in the correct answer may be visualized in augmented reality and displayed on the output unit according to a user's selection.

The AI tutoring unit classifies one side as a reactant and the other side as a product based on an arrow or an equal sign in the chemical reaction formula; Regarding the positional relationship between the chemical symbol and the number, (i) when the number is arranged in front of the chemical symbol, (ii) when the number is arranged in a subscript after the chemical symbol, and (iii) ) The context of the chemical reaction formula can be extracted by distinguishing the case where the number is placed as a superscript at the rear of the chemical symbol.

The AI tutoring unit distinguishes a reaction product composed of one or more substances from the chemical reaction formula and a product composed of one or more substances, and selects a reference reaction formula having the highest degree of agreement from the database based on the substances included in the chemical reaction formula; formulas of reactants and formulas of products for the chemical reaction formula are compared with chemical formulas of reactants and chemical formulas of products for the reference reaction formula, respectively, to extract formula errors; After correcting the chemical formula error, the coefficient of the reactant and the coefficient of the product for the chemical reaction equation may be compared with the coefficient of the reactant and the product coefficient for the reference reaction equation, respectively, to extract the coefficient error.

The AR implementation unit detects a chemical term described in the object displayed on the output unit (the chemical term includes a chemical name and chemical formula); displaying a bounding box around the chemical term on the output unit when the chemical term coincides with the reference term; When a selection event occurs for the bounding box, a selection is selected from the group consisting of the 'Reading method' option window, the 'View molecular formula' option window, the 'View structural formula' option window, and the 'View additional information' option window for the chemical term. displaying at least one option window on the output unit; When a selection event occurs for any one of the option windows, education information corresponding to the selected option window may be displayed on the output unit.

Details of other embodiments are included in the specification and drawings.

As described above, according to the chemistry education device according to the present invention and the method of providing chemistry education service using the same, when the chemistry education device photographs an object such as paper, chemical terms or chemical reaction formulas among handwritten or printed text on the object can be automatically detected, and AI tutoring service or AR implementation service can be provided according to the user's choice.

In the case of the AI tutoring service, chemical symbols, numbers, and other symbols can be individually recognized from the detected chemical reaction formula, and the context of the chemical reaction formula can be extracted using the mutual positional relationship between them. Chemical formula errors and coefficient errors can be sequentially and automatically extracted by comparing the chemical reaction formula with a reference reaction formula previously defined in the database. By displaying a bounding box around the error on the output unit and displaying the correct answer to the error and its chemical structure in augmented reality around the bounding box, the user can easily identify and understand the error in the chemical reaction formula.

In the case of an AR implementation service, one or more chemical terms are detected on an object and a bounding box is displayed around the detected chemical terms on an output unit. ' Option window, 'View molecular formula' option window, 'View structural formula' option window, 'View additional information' option window, or a combination thereof may be displayed on the output unit. When the user selects one of these option windows, educational information corresponding to the selected option window is displayed on the output unit. For example, when a chemical term is composed of a predetermined chemical formula, the chemical name of the chemical formula is displayed on the output unit when the user selects the 'Reading Method' option window. When the chemical term is composed of a predetermined chemical name, the molecular formula of the chemical name is displayed on the output unit when the user selects the 'View Molecular Formula' option window. In addition, if the user selects the 'View Structure' option window, the structural formula of the chemical formula or chemical name is displayed in the output section, and if the user selects the 'View Additional Information' option, additional information about the chemical formula or chemical name, such as the web A site or video can be driven. Therefore, when a user encounters an unfamiliar chemical term while studying chemistry, he or she can automatically and easily obtain useful information for learning simply by taking a picture of an object written with the chemical term with the chemistry education device.

1 is a configuration diagram conceptually showing a chemistry education device according to an embodiment of the present invention.
2 is a configuration diagram conceptually showing the AI tutoring unit of FIG. 1 .
3 is a flowchart illustrating a method of providing an AI chattering service according to an embodiment of the present invention.
4 is a diagram showing a process of detecting a chemical reaction formula using an artificial intelligence algorithm by way of example.
5 is a conceptual configuration diagram of the AR implementation unit of FIG. 1 .
6 is a flowchart illustrating a method of providing an AR implementation service according to an embodiment of the present invention.
7 is a diagram showing a process of detecting a chemical term using an artificial intelligence algorithm as an example.
8 is a diagram illustrating a process of providing an AI tutoring service using a chemistry education device according to an embodiment of the present invention.
9 is a diagram illustrating a process of providing an AR implementation service using a chemistry education device according to an embodiment of the present invention.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms, only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention belongs. It is provided to fully inform the holder of the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numbers designate like elements throughout the specification.

The "chemical formula" used in the present invention can be understood as a concept including molecular formulas, empirical formulas, rational formulas, and structural formulas commonly used in the chemical field. there is. In addition, the "chemical term" used in the present invention may be understood as a concept including a chemical formula and a chemical name.

Hereinafter, a chemistry education device according to an embodiment of the present invention and a method of providing a chemistry education service using the same will be described in detail with reference to the accompanying drawings. 1 is a configuration diagram conceptually showing a chemistry education device according to an embodiment of the present invention.

Referring to Figure 1, the chemistry education device 100 is a device that provides a chemistry education service, using artificial intelligence and augmented reality to correct errors in chemical reactions <AI tutoring service> or educational information on chemical terms It is a device that selectively provides the provided <AR implementation service>. Specifically, the chemistry education device 100 includes a camera unit 10, an input unit 20, an output unit 30, an AI tutoring unit 40, an AR implementation unit 50, and a database 60.

The camera unit 10 displays image information generated by photographing an object such as paper, a book, or printed matter on the output unit 30 .

The input unit 20 includes a software or hardware input device, and the output unit 30 includes a display. The display may include a user interface as a means for detecting a user's touch input in UI/UX of operating system software and UI/UX of application software. The display may include a touch screen that simultaneously executes a function of an output unit that outputs a screen and a function of an input unit that detects a user's touch event. The output unit 30 outputs image information of the object to a display screen.

The AI tutoring unit 40 detects a chemical reaction formula written on the object from image information of the object and provides an artificial intelligence tutoring service for the chemical reaction formula.

The AR implementation unit 50 detects a chemical term written on the object from image information of the object and provides an augmented reality implementation service for the chemical term.

Database 60 generally provides a place to store computer code and data used by devices. Specifically, the database 60 defines and stores a plurality of standard reaction formulas for AI tutoring services and a plurality of standard terms for AR implementation services in advance. The database 60 includes not only mobile applications and resources required for their operation/management, but also basic input/output systems, operating systems, various programs, applications, user interface functions executed in devices, processor functions, etc. Firmware for the device may be stored.

The chemistry education device 100 may include a separate processor (not shown), and the processor executes computer codes along with an operating system and generates and uses data. The processor may also receive and process input and output data between components of the chemistry education device 100 using a series of instructions. In addition, the processor serves as a control unit that executes functions of operating system software and various application software installed in the chemistry education device 100 .

The chemistry education device 100 according to an embodiment of the present invention may include a terminal device having a wireless communication function that a user can move and carry. For example, such a terminal may be a mobile communication terminal device capable of communication access through a mobile communication network, a navigation terminal device, a smart phone, a tablet PC, a wearable smart device, a PDA, a computer, a laptop computer, and the like.

Referring to FIGS. 2 to 4, an AI tutoring unit and a method of providing an AI tutoring service using the same according to an embodiment of the present invention will be described in detail. 2 is a configuration diagram conceptually showing the AI tutoring unit of FIG. 1 . 3 is a flowchart illustrating a method of providing an AI chattering service according to an embodiment of the present invention. 4 is a diagram showing a process of detecting a chemical reaction formula using an artificial intelligence algorithm by way of example.

2 and 3, the AI tutoring unit 40 includes a chemical reaction detection unit 42, a matching unit 44, an error correction unit 46 and a modeling unit 48, <AI tutoring service > proceeds as follows.

First, when the camera unit 10 photographs an object to generate image information and displays it on the output unit 30 (S31), the chemical reaction detector 42 outputs the image information of the object displayed on the output unit 30. A chemical equation described on the object is sensed from (S32).

The chemical reaction formula detection unit 42 classifies and recognizes chemical symbols, numbers, and other symbols constituting the chemical reaction formula through artificial intelligence and machine learning (S33). In detail, the chemical reaction detector 42 recognizes text from an object image using a machine learning engine, and detects a token probability of a LaTeX code from a softmax layer of the machine learning engine. Subsequently, the chemical reaction detector 42 fetches a LaTeX markup command from the most probable token, and converts the LaTeX command into a MathML (Mathematical Markup Language) version. Subsequently, the chemical reaction formula detection unit 42 stores MathML in the database 60, and distinguishes chemical symbols, numbers, and other symbols from the character strings constituting the chemical reaction formula and recognizes them.

The chemical reaction formula of this embodiment may be composed of chemical symbols, numbers, and other symbols, and the chemical reaction formula detection unit 42 extracts the context of the chemical reaction formula by using the positional relationship between chemical symbols, numbers, and other symbols (S34). . Here, the chemical symbol is a symbol that is written with one or two Roman letters to simply indicate a chemical element. For example, hydrogen is a chemical symbol such as H, oxygen is O, and iron is Fe. can indicate In addition, the chemical symbol can be understood as substantially the same as the element symbol on the periodic table of elements.

The number can represent a stoichiometric coefficient, number of elements, or number of ions. When a number is placed in front of a chemical symbol, the number can be understood to have the meaning of a coefficient or stoichiometric coefficient. When a number is placed as a subscript after a chemical symbol, it can be understood that the number has the meaning of the number of elements. When a number is placed in a superscript after a chemical symbol and written with a negative or positive sign, the number can be understood to have the meaning of the number of ions.

Other symbols may include arrows, equal signs (=), positive signs (+), negative signs (-), and the like. In the character string constituting the chemical reaction equation, one side may be classified as a reactant and the other side as a product based on an arrow or an equal sign.

In this way, the chemical reaction detection unit 42 first classifies one side as a reactant and the other side as a product based on other symbols (eg, arrows or equal signs) in the chemical reaction equation, and then determines the positional relationship between the chemical symbol and the number. In this case, (i) numbers are placed in front of chemical symbols, (ii) numbers are placed in subscripts after chemical symbols, and (iii) numbers are placed in superscripts after chemical symbols. Separate and extract the context of the chemical reaction formula.

Subsequently, the matching unit 44 compares the chemical reaction formula with a reference reaction formula predefined in the database 60 to extract an error (S35). Specifically, the matching unit 44 sequentially performs <process of selecting standard reaction equation>, <process of extracting chemical formula error>, and <process of extracting counting error>.

Looking at <Process of Selecting Standard Reaction Equation>, the matching unit 44 distinguishes a reactant and a product from the detected chemical equation. Here, the reactants may consist of one or more substances, and the products may consist of one or more substances. The matching unit 44 searches for the reference reaction formula stored in the database 60 based on the substances included in the chemical reaction formula, and selects the reference reaction formula with the highest degree of agreement from the database 60.

Looking at the <Extraction Process of Chemical Formula Error>, the matching unit 44 extracts a chemical formula of a reactant and a chemical formula of a product in a chemical reaction formula. When a reactant or product consists of a plurality of substances, the chemical formula of each substance is extracted. The matching unit 44 compares chemical formulas of reactants and chemical formulas of products for chemical reaction formulas with chemical formulas of reactants and chemical formulas of products for a predefined reference reaction formula, respectively, to extract formula errors.

Looking at <Extraction Process of Coefficient Error>, after correcting the chemical formula error, the matching unit 44 calculates the coefficients of reactants and products for the chemical reaction equation, and the coefficients of reactants and products for the reference reaction equation. , respectively, to extract coefficient errors.

Referring to the example below, the error extraction process can proceed as follows.

[Chemical Reaction Equation] 2H ₂ + O ₃ → H ₂ O

[Standard Reaction Equation] 2H ₂ + O ₂ → 2H ₂ O

[Chemical formula error extraction] 2H ₂ + O ₂ → H ₂ O

[Extract count error] 2H ₂ + O ₂ → 2H ₂ O

First, the matching unit 44 distinguishes between reactants (H ₂ and O ₃ ) and products (H ₂ O) in the chemical reaction formula (2H ₂ + O ₃ → H ₂ O). The matching unit 44 selects the reference reaction formula (2H ₂ + O ₂ → 2H ₂ O) having the highest degree of coincidence based on the three materials (H ₂ , O ₃ , H ₂ O) included in the chemical reaction formula. The standard reaction formula and chemical reaction formula selected in this example coincide with the two materials (H ₂ , H ₂ O). Subsequently, the matching unit 44 converts the chemical formula (H ₂ , O ₃ ) of the reactant and the chemical formula (H ₂ O) of the product to the chemical reaction formula, and the chemical formula (H ₂ , O ₂ ) of the reactant and the chemical formula of the product to the reference reaction formula. (H ₂ O), respectively. In this example, an error is found in the chemical formula (O ₃ ) of the reactant in the chemical reaction equation. Subsequently, the matching unit 44 corrects the chemical formula error and extracts the coefficient error by comparing the coefficient of the chemical reaction equation with the coefficient of the reference reaction equation. In this example, an error is found in the coefficient of the product (H ₂ O) of the chemical reaction.

Referring back to FIGS. 2 and 3 , the error correction unit 46 displays a bounding box around the error on the output unit 30 when there is an error in the chemical reaction equation (S36).

The error correction unit 46 displays the correct answer for the error on the output unit 30 (S37). Here, in the case of a correct answer, it may be made in the form of text such as a chemical formula.

According to the user's selection, the modeling unit 48 visualizes the chemical structural formula in the correct answer in 2D or 3D augmented reality and displays it on the output unit 30 (S38).

Referring to FIG. 4, the chemical reaction formula detection process of the AI tutoring unit according to an embodiment of the present invention will be described.

A convolutional neural network (CNN) unit extracts a feature from an input image of a chemical reaction using a convolution layer, a pooling layer, and an activation layer (S41). In each convolution layer, the input image is convoluted with a series of kernels or image filters. The pooling layer reduces the image size and increases the size of the receptive field. Activation layers add non-linearity to neural networks.

The positional encoding unit identifies positional relationships between chemical symbols, numbers, and other symbols in the chemical reaction formula to extract important mathematical meanings or contexts of the chemical reaction formula (S42). This positional relationship can be extended in various directions such as left and right, up and down, subscript, superscript, and overlapping.

The context of the location encoding unit and the feature map of the CNN unit are combined together and unfolded into a one-dimensional array to form an encoder output (S43).

Since the chemical equation tokens are related to each other, a word embedding layer generally used in natural language processing (NLP) is added to project the tokens into high-dimensional vectors (S44). These embeddings capture the interrelationships between different tokens.

The decoder model is based on bidirectional long-short term memory (Bi-LSTM) cells and captures more complex language semantics (S45). Here, bidirectional long-short-term memory can handle longer sequences better than standard recurrent neural networks (RNNs). Using bidirectional cells in each measurement helps capture context in both forward and backward directions.

Long short-term memory (LSTM) can be extended to capture long-term memory, but it is not uncommon for the markup of a complex chemical reaction to extend to 100 or more tokens. In this case, the initial hidden state vector of the RNN is not sufficient to compress all the information of the encoder. In addition, since the CNN encoder does not have a memory function, an attention mechanism is introduced to solve this problem (S46). The Attention layer uses the encoder output to create a context vector.

A softmax function is used to generate the probability distribution of the next token for the vocabulary (S47).

Referring to FIGS. 5 to 7 , an AR implementation unit according to an embodiment of the present invention and a method of providing an AR implementation service using the same will be described in detail. 5 is a conceptual configuration diagram of the AR implementation unit of FIG. 1 . 6 is a flowchart illustrating a method of providing an AR implementation service according to an embodiment of the present invention. 7 is a diagram showing a process of detecting a chemical term using an artificial intelligence algorithm as an example.

5 and 6, the AR implementation unit 50 includes a chemical term detection unit 52, a matching unit 54, and an option execution unit 56, and <AR implementation service> proceeds as follows. do.

First, when the camera unit 10 photographs an object to generate image information and displays it on the output unit 30 (S61), the chemical term detector 52 outputs the image information of the object displayed on the output unit 30. A chemical term written on the object is sensed from (S62). Here, chemical terms may include chemical formulas and chemical names.

The chemical term detecting unit 52 recognizes chemical formulas and/or chemical substance names through artificial intelligence and machine learning (S63). Specifically, the chemical term detection unit 52 recognizes text from an object image using a machine learning engine, and detects a token probability of a LaTeX code from a softmax layer of the machine learning engine. Subsequently, the chemical term detection unit 52 fetches a LaTeX markup command from the most probable token, and converts the LaTeX command into a MathML (Mathematical Markup Language) version. Subsequently, the chemical term detecting unit 52 stores MathML in the database 60, and classifies and recognizes chemical formulas or chemical names.

The matching unit 54 compares the chemical term with a standard term predefined in the database 60, and displays a bounding box around the chemical term on the output unit 30 when the chemical term matches the standard term ( S64). If there are reference terms matching the plurality of chemical terms in the database 60, a bounding box may be displayed around each of the plurality of chemical terms.

When a selection event by the user occurs for any one of the bounding boxes displayed on the output unit 30, the option execution unit 56 displays a 'reading method' option window for the selected chemical term, a 'view molecular formula' option window, At least one option window selected from the group consisting of the 'view structure' option window and the 'view additional information' option window is displayed on the output unit 30 (S65).

When a selection event occurs for any one of the option windows by the user, the option execution unit 56 displays educational information corresponding to the selected option window on the output unit 30 (S66). When the user selects the 'reading method' option window, the method for reading the chemical formula within the selected bounding box or the chemical name is displayed on the output unit 30 . When the user selects the 'view molecular formula' option window, the molecular formula corresponding to the chemical name within the selected bounding box is displayed on the output unit 30. When the user selects the 'view structural formula' option window, the structural formula corresponding to the chemical formula or chemical name within the selected bounding box is displayed on the output unit 30. When the user selects the 'view additional information' option window, additional information corresponding to the chemical formula or chemical name within the selected bounding box, for example, an encyclopedia website or video, is displayed on the output unit 30 .

Referring to FIG. 7 , a chemical term detection process of the AR implementation unit according to an embodiment of the present invention will be described.

When the input image of the object is supplied to the CNN layer, the CNN layer extracts a related feature from the image (S71). Specifically, the convolution operation extracts features from an image by applying filter kernels of various sizes. And a nonlinear ReLU (rectified linear unit) function is applied. Pooling layers summarize image regions and output reduced versions of their inputs.

Subsequently, the feature sequence provided from the CNN layer is supplied to the RNN cell that stores related information through this sequence (S72). A long short-term memory (LSTM) implementation of RNN is used. It can propagate information over long distances and provide more powerful training properties than vanilla RNNs. RNN output sequences are mapped to matrices encoding temporal information.

The RNN output matrix and ground truth text are provided to CTC (connectionist temporal classification), and CTC calculates a loss value (S73). During reasoning, only the matrix is provided to the CTC and the CGTC decodes it into the final text (S74). Therefore, in the above process, CTC calculates a loss value for training a neural network (NN), decodes a matrix to get text included in an input image, and infers a handwritten text label.

8 is a diagram illustrating a process of providing an AI tutoring service using a chemistry education device according to an embodiment of the present invention. 9 is a diagram illustrating a process of providing an AR implementation service using a chemistry education device according to an embodiment of the present invention. 8 and 9 illustrate the case of using a smartphone as an example of a chemistry education device.

Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art can realize that the present invention can be implemented in other specific forms without changing the technical spirit or essential features. you will be able to understand Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting.

Claims (8)

As a method of providing a chemistry education service using a chemistry education device including a camera unit, an output unit and a database,
Optionally providing an AI tutoring service or an AR implementation service by the chemistry education device,
The step of providing the AI tutoring service,
photographing the object and displaying image information of the object on the output unit;
detecting a chemical reaction formula described on the subject;
Recognizing chemical symbols, numbers and other symbols constituting the chemical reaction formula;
extracting the context of the chemical reaction formula by using the positional relationship between the chemical symbol, the number, and the other symbols;
Defining a plurality of reference reaction formulas in advance in the database, and extracting errors by comparing the chemical reaction formula with the reference reaction formula;
If there is an error in the chemical reaction equation, displaying a bounding box around the error on the output unit; and
Displaying a correct answer to the error on the output unit and visualizing a chemical structural formula in the correct answer on the output unit in augmented reality according to a user's selection. According to claim 1,
Extracting the context,
Classifying one side as a reactant and the other side as a product based on an arrow or an equal sign in the chemical reaction equation; and
Regarding the positional relationship between the chemical symbol and the number, (i) when the number is arranged in front of the chemical symbol, (ii) when the number is arranged in a subscript after the chemical symbol, and (iii) ) A method of providing a chemistry education service comprising the step of extracting the context of the chemical reaction formula by identifying a case in which the number is placed in a superscript at the rear of the chemical symbol. According to claim 1,
The step of extracting the error is,
Distinguishing a reactant made of one or more materials from the chemical reaction formula and a product made of one or more materials, and selecting a reference reaction formula having the highest degree of agreement based on the materials included in the chemical reaction formula from the database;
extracting formula errors by comparing chemical formulas of reactants and chemical formulas of products for the chemical reaction formula with chemical formulas of reactants and chemical formulas of products for the reference reaction formula, respectively; and
After correcting the chemical formula error, the coefficient of the reactant and the coefficient of the product for the chemical reaction formula are compared with the coefficient of the reactant and the coefficient of the product for the reference reaction formula, respectively, and extracting the coefficient error. How to provide. According to claim 1,
The step of providing the AR implementation service,
photographing the object and displaying it on the output unit;
detecting a chemical term written on the subject (the chemical term includes a chemical name and chemical formula);
predefining a plurality of reference terms in the database, and displaying a bounding box around the chemical terms on the output unit when the chemical terms coincide with the reference terms;
When a selection event occurs for the bounding box, a selection is selected from the group consisting of the 'Reading method' option window, the 'View molecular formula' option window, the 'View structural formula' option window, and the 'View additional information' option window for the chemical term. displaying at least one option window on the output unit; and
When a selection event occurs for any one of the option windows, displaying education information corresponding to the selected option window on the output unit. As a chemistry education device that provides chemistry education services,
a camera unit that photographs an object;
an output unit displaying image information of the object;
an AI tutoring unit providing an AI tutoring service for the chemical reaction described on the subject;
an AR implementation unit providing an AR implementation service for chemical terms written on the object; and
Including a stored database that defines a plurality of reference reaction formulas and a plurality of reference terms in advance,
The AI tutoring unit,
detecting a chemical reaction formula described in the object from the image information displayed on the output unit;
Recognize chemical symbols, numbers and other symbols constituting the chemical reaction formula;
Extracting the context of the chemical reaction equation using positional relationships between the chemical symbol, the number, and the other symbols;
extracting errors by comparing the chemical reaction equation with the reference reaction equation;
if there is an error in the chemical equation, display a bounding box around the error on the output unit;
Chemistry education device, characterized in that for displaying the correct answer to the error on the output unit and visualizing the chemical structural formula of the correct answer on the output unit in augmented reality according to the user's selection. According to claim 5,
The AI tutoring unit,
Classifying one side as a reactant and the other side as a product based on an arrow or an equal sign in the above chemical reaction formula;
Regarding the positional relationship between the chemical symbol and the number, (i) when the number is arranged in front of the chemical symbol, (ii) when the number is arranged in a subscript after the chemical symbol, and (iii) ) Chemistry education device, characterized in that for extracting the context of the chemical reaction formula by distinguishing the case where the number is placed in a superscript at the rear of the chemical symbol. According to claim 5,
The AI tutoring unit,
distinguishing a reactant composed of one or more substances from the chemical reaction formula and a product composed of one or more substances, and selecting a reference reaction formula having the highest degree of agreement based on the materials included in the chemical reaction formula from the database;
formulas of reactants and formulas of products for the chemical reaction formula are compared with chemical formulas of reactants and chemical formulas of products for the reference reaction formula, respectively, to extract formula errors;
After correcting the chemical formula error, the coefficient of the reactant and the coefficient of the product for the chemical reaction equation are compared with the coefficient of the reactant and the coefficient of the product for the reference reaction equation, respectively, to extract the coefficient error. Chemistry education device. According to claim 5,
The AR implementation unit,
detecting a chemical term described in the object displayed on the output unit (the chemical term includes a chemical name and chemical formula);
displaying a bounding box around the chemical term on the output unit when the chemical term coincides with the reference term;
When a selection event occurs for the bounding box, a selection is selected from the group consisting of the 'Reading method' option window, the 'View molecular formula' option window, the 'View structural formula' option window, and the 'View additional information' option window for the chemical term. displaying at least one option window on the output unit;
When a selection event occurs for any one of the option windows, the chemistry education device, characterized in that for displaying educational information corresponding to the selected option window on the output unit.

Images