JPWO2019166967A5 - - Google Patents

Description

Technical Field The present invention relates to the field of computers, and more particularly to processing units for heterogeneous data.

Background High-performance computing and supercomputing have supported many achievements, including the discovery of the Higgs boson (CERN) and the sequencing of the human genome (Human Genome Project). High performance computing (HPC) and supercomputing are expected to have significant impacts in areas such as medicine and medicine, modeling and simulation, security, fintech, engineering and manufacturing.

The HPC and supercomputing communities face challenges that need to be addressed in order to have a deeper impact on the above industries. Some challenges are to handle scalability at both the hardware and software levels, represent and process multi-level hierarchies, achieve heterogeneous data fusion, and find alternative ways to compensate for human cognitive weaknesses. It is to deal with the human bottleneck problem. The capabilities of prioritized processors and computer systems need to be further developed to handle exascale and above computing, and are functionally easy to use and understand so that researchers and the industry can take advantage of supercomputers. We also need a complete common tool.

In addition, numerous systems have been developed to run computer models or simulations. These systems are usually limited to a few applications and in most cases can only be used in one area of knowledge. As a result, system experts are not proficient in other systems unless considerable effort is spent in training and learning. In addition, models and simulation models are usually incompatible between platforms. While some companies offer some compatibility within their product ecosystem, users often need to know each product and understand their methods, methods, and modeling paradigms. .. This is an important drawback.

Some simulation software or solutions provide access to software development kits (SDKs) or scripting languages. These scripting languages are usually text and resemble programming languages such as C ++®, C #®, Java® or Python®. With such software languages, users quickly run into what could be called "coding barriers," which significantly limits the types of users who can use the software. To make the software more accessible, the sources of these software may hide the code, functions, and scripts in blocks that appear as symbols in the 2D visual interface. A block diagram is created by adding and interconnecting multiple blocks, and the block diagram represents a simulation model presented in 2D. After running the simulation, the output can be displayed in a 2D graph for analysis. In such a system, there is a gap between the representation of the physical model and the data associated with the physical model. For example, you can access and graph the forces acting on the wheels of a car, but there is no 3D model of a car with rotating wheels. In the automotive industry, various components can be interconnected and simulated, but these simulations will be discarded when the vehicle is designed. Such simulations are intended to study abstract information about the product and rely primarily on differential equations. Since users require advanced knowledge of physics and mathematics, such programming systems are used only by trained humans such as engineers or researchers.

Modern science and corporate projects are now interdisciplinary, collaborative, global and highly complex. There are many large-scale interdisciplinary project initiatives that reorganize numerous companies and public bodies (Visual Physiological Human Project, Human Brain Project, Brain Initiative). In such projects, only a small proportion of the participants have the skills to create models and simulations. Most participants are experts in their own highly specialized field, and it is difficult to convey their knowledge to experts in other fields or to integrate their knowledge into a large cohesive model. be.

Therefore, there is a need for an improved processing unit architecture that allows for improved programming methods.

Overview According to the first broad aspect, a processing unit device, at least one control unit that controls the operation of the processing unit device, and at least one conversion associated with a conversion performed by at least one control unit. A transformational logical unit containing blocks, the transformation containing effects applied to the output site contained within the output universe, each of at least one conversion block containing an effect block and an output site block, where the effect block is an effect. The output site block comprises a conversion logic unit that includes at least one second storage unit for storing information about the output site. A processing unit device is provided.

In one embodiment, at least one control unit is configured to perform further conversions using a form conversion instruction set containing transformation data defining parameters for the transformation.

In one embodiment, at least one control unit is included in the conversion logic unit.

In one embodiment, the at least one control unit is contained in at least one of the conversion blocks.

In one embodiment, at least one control unit comprises a plurality of control units, at least one conversion block comprises a plurality of conversion blocks, and each of the plurality of control units is a conversion block of the plurality of conversion blocks. include.

In one embodiment, the at least one control unit includes at least one first logical unit contained in the effect block and at least one second logical unit contained in the output site block.

In one embodiment, the at least one first logic unit and the at least one second logic unit include one of an arithmetic logical operation unit and a logic gate.

In one embodiment, the at least one control unit further comprises a central unit included in the conversion logic unit.

In one embodiment, at least one control unit can be connected to external memory.

In one embodiment, information about the output universe is stored in external memory.
In one embodiment, the processing unit device further comprises internal memory.

In one embodiment, information about the output universe is stored in internal memory.
In one embodiment, information about the output universe is stored in at least one of some of the first storage units and some of the second storage units.

In one embodiment, at least one first storage unit and at least one second storage unit include registers.

In one embodiment, each of the at least one conversion block further comprises an input site block comprising at least one third storage unit for storing information about the input sites contained in the input universe.

In one embodiment, the at least one control unit comprises a plurality of control units and the input site block comprises at least one plurality of control units.

In one embodiment, each of the at least one conversion block
An input site range block with at least one fourth storage unit for storing information about the input range that defines the subspace of the input universe.
It further comprises an output site range block comprising at least one fifth storage unit for storing information about an output range that defines a subspace of the output universe.

According to another broad aspect, a processing unit device that comprises at least one control unit that controls the operation of the processing unit device and at least one conversion block that is associated with the conversion performed by at least one control unit. A transformational logic unit that contains a transformation that contains effects that are modified by information from the input site contained in the input universe, each of which at least one transformation block contains an effect block and an input site block. The input site block comprises a conversion logic unit that includes at least one second storage unit for storing information about the effect and at least one second storage unit for storing information about the input site. , Processing unit device is provided.

In one embodiment, at least one control unit is configured to perform further conversions using a form conversion instruction set containing transformation data defining parameters for the transformation.

In one embodiment, at least one control unit is included in the conversion logic unit.

In one embodiment, the at least one control unit is contained in at least one of the conversion blocks.

In one embodiment, at least one control unit comprises a plurality of control units, at least one conversion block comprises a plurality of conversion blocks, and each of the plurality of control units is a conversion block of the plurality of conversion blocks. include.

In one embodiment, the at least one control unit includes at least one first logical unit contained in the effect block and at least one second logical unit contained in the input site block.

In one embodiment, the at least one first logic unit and the at least one second logic unit include one of an arithmetic logical operation unit and a logic gate.

In one embodiment, the at least one control unit further comprises a central unit included in the conversion logic unit.

In one embodiment, at least one control unit can be connected to external memory.

In one embodiment, information about the input universe is stored in external memory.
In one embodiment, the processing unit device further comprises internal memory.

In one embodiment, information about the input universe is stored in internal memory.
In one embodiment, information about the input universe is stored in at least one of some of the first storage units and some of the second storage units.

In one embodiment, at least one first storage unit and at least one second storage unit include registers.

In one embodiment, each of the at least one conversion block further comprises an output site block containing at least one third storage unit for storing information about the output sites contained in the output universe.

In one embodiment, the at least one control unit comprises a plurality of control units and the output site block comprises at least one plurality of control units.

In one embodiment, each of the at least one conversion block
An input site range block with at least one fourth storage unit for storing information about the input range that defines the subspace of the input universe.
It further comprises an output site range block comprising at least one fifth storage unit for storing information about an output range that defines a subspace of the output universe.

According to a broader aspect, a method performed by a computer for programming to receive a transformation that includes at least one output site and an effect applied to the form that identifies the form contained in the output universe. And a step to retrieve the output match with the output site from the output universe, and a step to apply the effect to the retrieved output match, resulting in a modified output match, an applied step, and a modified output. A method performed by a computer is provided, including a step to output a match.

In one embodiment, the step of outputting the modified output match comprises displaying the output universe and the graphic display of the modified output match within the output universe.

In one embodiment, the method has been modified to further include displaying a user graphical interface that includes a first section for representing the output universe and a second section for displaying a graphic representation of the transformation. The above step of outputting an output match includes displaying the modified output match within the first section of the user graphical interface.

In one embodiment, the transformation further comprises transformation data defining the parameters of the transformation, and the step of applying the effect to the retrieved output match is performed using the transformation data.

In one embodiment, the transformation further comprises an input site, the method further comprises a step of retrieving an input match with the input site from the input universe, and the above step of applying an effect to the retrieved output match is retrieved. It is performed using the information related to the input match.

In one embodiment, the method provides a user graphical interface that includes a first section for representing the output universe, a second section for defining effects, and a third section for representing the input universe. Includes more steps to display.

In one embodiment, the method is
Steps to display the input site in the third section of the graphical user interface,
Steps to display the graphic representation of the effect in the second section of the graphical user interface,
It further includes a step of displaying the output site in the first section of the graphical user interface.

In one embodiment, the method further comprises the step of receiving at least one of an input range and an output range, where the input range defines a subspace of the input universe and the output range is a subspace of the output universe. The input match is selected from the subspace of the input universe and the output match is selected from the subspace of the output universe.

In one embodiment, the method is to display a first section for representing the output universe, a second section for defining the effect, a third section for representing the input universe, and an output range. Also includes a step of displaying a user graphical interface that includes a fourth section of the universe, and a fifth section for displaying the input range.

In one embodiment, the method is
Steps to display the input site in the third section of the graphical user interface,
Steps to display the graphic representation of the effect in the second section of the graphical user interface,
Steps to display the output site in the first section of the graphical user interface,
Steps to display the graphic representation of the output range in the fourth section of the graphical user interface,
It further includes a step of displaying a graphic representation of the input range in the fifth section of the graphical user interface.

According to yet another broader aspect, a system for programming, the system is configured to perform the steps of the method described above, with the means of communication, the memory in which the statements and instructions are stored. A system is provided with a processing unit.

Yet another broad aspect provides a computer program product with computer-readable memory that stores computer-executable instructions that perform the steps of the above method when executed by a computer.

Also according to a broader aspect, a computer-performed method for programming, a step of receiving a transformation that includes at least one input site and an effect applied to the form, and an input universe. A step to retrieve an input match with an input site from, and a step to modify the effect using the information associated with the input match, resulting in a modified effect, a modified step, and a modified effect. A method performed by a computer is provided, including steps to output.

In one embodiment, the step of outputting the modified effect comprises displaying a graphic display of the modified effect.

In one embodiment, the method has been modified to further include displaying a user graphical interface that includes a first section for representing the input universe and a second section for displaying the graphic representation of the transformation. The above steps of outputting an effect include displaying the modified effect within a second section of the user graphical interface.

In one embodiment, the transformation further comprises transformation data defining the parameters of the transformation, and the above steps of modifying the effect are performed using the transformation data.

In one embodiment, the transformation further comprises an output site contained within the output universe.
In one embodiment, the method comprises a first section for representing the output universe, a second section for representing the modified effect, and a third section for representing the input universe. Includes additional steps to display the graphical interface.

In one embodiment, the method, in one embodiment, the method,
Steps to display the input site in the third section of the graphical user interface,
Steps to display the graphic representation of the modified effect in the second section of the graphical user interface,
It further includes a step of displaying the output site in the first section of the graphical user interface.

In one embodiment, the method further comprises the step of receiving at least one of an input range and an output range, where the input range defines a subspace of the input universe and the output range is a subspace of the output universe. And the input match is selected from the subspace of the input universe.

In one embodiment, the method comprises a first section for representing the output universe, a second section for representing the modified effect, a third section for representing the input universe, and an output range. It further comprises a step of displaying a user graphical interface that includes a fourth section for displaying and a fifth section for displaying the input range.

In one embodiment, the method is
Steps to display the input site in the third section of the graphical user interface,
Steps to display the graphic representation of the modified effect in the second section of the graphical user interface,
Steps to display the output site in the first section of the graphical user interface,
Steps to display the graphic representation of the output range in the fourth section of the graphical user interface,
It further includes a step of displaying a graphic representation of the input range in the fifth section of the graphical user interface.

According to yet another embodiment, it is a system for programming, in which the communication means, the memory in which the statements and instructions are stored, and the processing configured to perform the steps of the method described above. A system with units is provided.

Yet another embodiment provides a computer program product comprising computer readable memory for storing computer executable instructions that perform the method steps of the above method when executed by a computer.

For the purposes of this specification, a form is an object such as a 2D object or a 3D object, text, formula, transformation, collection of forms, collection of transformations, collection of forms and transformations, object representations, sets, collections of sets. , Animated sequences, etc., may be understood as an element or a collection of elements. Examples of forms include representations of cars, biological cats, molecules and atoms, tree collections, symbols, transformations, and so on. It should be understood that a collection of forms is also a form. Other examples of forms are 3D meshes in 3D or 4D coordinate systems, multidimensional vectors, tensors, artificial neural networks, artificial neural networks represented in 2D or 3D, tables, arrays, boxels in 3D coordinate systems, shapes. And continuous structures such as solid lines, physical objects, symbols, symbols consisting of pixels, words, scripts, codes, numbers, functions, algorithms, equations, neural networks, any type of data (eg graphics, visuals, sounds, etc.) Digital, image, text, distance, measure, time, ...), data structure, file name, z-buffer content, storage content, storage address, topology, memory address, form object, form object name, tag with variables or Includes representations, sets of form objects, transformations, transformation names, representations with their variables and sets of transformations, and so on.

The input universe should be understood as a collection of forms associated with the input site. The input site should be understood as at least one form selected within the input universe. The input range should be understood as a subspace of the input universe from which the transformation can obtain data. That is, it limits the extraction of information to a particular subset / part of the input universe. This subset / part of the input universe can be represented as forms, sets (using union operators, negation operators and AND operators), collections, regions, intervals, inequalities and equations. For inequalities and equations, it is understood that a collection of their solutions constitutes a subset / part of the input universe.

The output universe should be understood as a collection of forms associated with the output site. The output site should be understood as at least one form that needs to be selected within the output universe. Think of the output site as an indication of where in the output universe the effect associated with the transformation should be applied, or as an identification of the form (s) of the output universe that needs to be replaced, added, then added, or modified. be able to. The output range should be understood as a subspace of the output universe to which the transformation can be applied. That is, it limits the application of the transformation to a particular subset / part of the output universe. This subset / part of the output universe can be represented as forms, sets (using union operators, negation operators and AND operators), collections, regions, intervals, inequalities and equations. For inequalities and equations, it is understood that a collection of their solutions constitutes a subset / part of the input universe.

An effect should be understood as a completely new form, or a form consisting of an input site (s), an output site, or a part of an input site and an output site (s). For example, an effect can change the characteristics of a form defined at the output site, such as changing the shape of the form. In another example, the effect can add a form to the form defined at the output site.

The transformation data refers to a set of information / parameters (number of times, name, collection of forms, etc.) used for the effect associated with the transformation.

The conversion consists of at least an effect and an output site (which can optionally include an output range) and / or an input site (which can optionally include an input range). The conversion can further include the conversion data. In one embodiment, the transformation consists of six parts: input range, input site, effect, output site, output range, and conversion data.

The input site is connected to the input universe and the output site is connected to the output universe. In one embodiment, the input universe can correspond to the output universe so that there is a single universe and both the input site and the output site are connected to the single universe.

In one embodiment, the method and system can be used to create a dynamic model consisting of a form to which a sequence of transformations and / or a parallel sequence of transformations is applied. A static or dynamic model can be thought of as the result of applying a sequence of transformations. It should be understood that any changes observed in the world can be modeled using this system and method.

It should be understood that the system is a collection of forms. In most cases, the system is, more precisely, a collection of forms with a collection of transformations applied to the form. In addition, the system is also considered to be a form.

In one embodiment, the system includes a timeline, and as a result, after applying the transformation, the system proceeds within the timeline. It should be understood that the timeline does not necessarily refer to time, but can also be understood as a step taken in applying the transformation. In one embodiment, the timeline can be synchronized with a real-world clock or virtual clock.

The conversion processing unit device and its architecture, which can be reproduced on a virtual machine, enable at least some of the following advantages over prior art processing equipment such as von Neumann processing equipment. These benefits are related to processing speed, energy consumption, storage size, computational accuracy, accuracy, fidelity and / or scalability. In addition, the conversion processing unit architecture provides new solutions to multiple challenges in high performance computing.

Interestingly, this transformation processing unit architecture based on transformation blocks can be used to simulate nondeterministic Turing machines. For example, changing a conversion block can trigger multiple other conversion blocks (or branches), resulting in other branches. This provides an example of how the conversion processing unit architecture can perform multithreading and each thread can be further divided into other threads.

Easily extend a conversion processing unit with multiple other conversion processing units without facing interconnect problems associated with heterogeneous devices due to its homogeneous structure (consisting almost exclusively of conversion blocks). Can be done. In this conversion processing unit architecture, expanding the computing power of a computer will connect more conversion processing units.

In one embodiment, the transmutation blocks result in the idea of an agent-based subprocessor in which each transmutation block is understood to act and react based on its environment and behave like an autonomous agent that contributes to global computational results. Can be created as follows.

In one embodiment, the architecture of the conversion processing unit device can be implemented by an internal clock.

In another embodiment, such as for asynchronous processors, the architecture of the conversion processing unit device provides clockless processing capabilities, which provide power efficiency, responsiveness, and robustness.

This architecture of the transformation processing unit device can be used in the context of machine learning and artificial neural networks. In the virtual machine implementation aspect of this transformation processing unit architecture, it has been shown that it is possible to create artificial neural networks and systems of neural networks that can be trained and used for inference. That is, machine learning algorithms can be designed and processed using transformations and forms, and thus can be deployed (or trained) directly on the processor.

Artificial neural networks on the processor can be used to achieve multiple internal optimizations of speed and storage space. In particular, on a processor, an automatic encoder can be used to compress data and deploy pre-trained artificial neural networks trained to represent or approximate complex functions or behaviors. Parts of the transformation processing unit can be used to achieve classifications that dispatch operations to various computational pipelines that allow matching processing and execution, or computational acceleration, artificial neural networks or machine learning algorithms. Can be reserved for expression. Furthermore, this also allows the transformation processing unit to process external data at high speed through various machine learning algorithms, not limited to neural networks.

In one embodiment, the form and transformation system can be designed to create a simulation. For example, the conversion processing unit device can be used to simulate a 3D beating heart by using the following conversions.
-Conversions that cause cells to contract (effect: contracted cells, output sites: cells), and-transformations that send information to cells adjacent to cells (effects: contracted cells + contracted adjacent cells, output sites: contracted cells) + Adjacent cells).

After cell contraction, the second transformation continuously propagates the contraction to all other connected cells, resulting in a heartbeat. Encoding the 3D data generated from the magnetic resonance image of the heart into the processor and running the processor with the transformation encoded according to the above principles provides a 3D heart simulation on the microchip.

The transformation can provide a data heterogeneity solution for some problems. Many modern microprocessors typically include one or more arithmetic and logical units and floating point units. These are intended to efficiently carry out numerical arithmetic calculations. The conversion processing unit performs calculations on forms and other types of structured data directly on the processor, allowing arithmetic operations to be performed in parallel or non-parallel. This allows structured data to be processed at processor speeds, allowing users or software to define new data types to run directly on the processor.

Due to their dynamic and homogeneous structure, transformations, and forms, just-in-time and pre-compilation can be performed by the transformation processing unit of the embodiments of the present invention. In particular, precompiling the form conversion instruction set allows you to create machine language that can be executed natively on the target machine, which has the advantages of faster execution and less disk space. Brought to you. Precompiling dynamic languages has been found to result in significant speedups (more than 16x). In addition, just-in-time compilation can be optimized by compiling frequently used parts of the code into machine language. The universal structure of transformations and forms allows such optimization opportunities to occur frequently during the transformation.

Since transformations and forms can represent models, methods, and calculations in any domain, multiple different models in different domains can be transformed into a single system of forms and transformations. Within this system, information between models (eg, from different scientific areas) can be exchanged at high speed. If each model is implemented outside of this architecture, a conversion step is required to transform the information each time it is exchanged between the models before exchanging the information, resulting in a significant increase in computation time. do. On the other hand, such a problem can be avoided while using the conversion processing unit architecture.

A major part of the execution time of a scientific program is usually spent processing loops. Loop transformations are known to improve the execution speed of various types of loops. This conversion processing unit conversion can be used to represent a loop. Compared to other programming languages, the code in which the conversion is written requires less space on memory or disk. In addition, the transformation itself can be modified by other transformations, thus rendering the instruction set to a higher order system. Therefore, the transformations can be modified in a similar way to the loop transformations, but with the advantage of allowing many new types of optimization and providing a way to control them. This is in contrast to the small amount of loop optimization normally provided by popular modern compilers.

In one embodiment, the transformation implements the design by using a local transformation and provides the opportunity to design the model based on a direct observation of a real-world system. In other words, transformations allow for efficient agent-based simulations where global effects emerge from localized rules. This is in contrast to the method of solving global (global) equations to determine solutions as well as local information and behavior.

For example, when designing a virtual heart, the current level of technology lies in using higher-order finite element methods and complex differential equations. Finding solutions to global (global) differential equations and using the finite element method requires costly calculations.

In reality, cardiomyocytes in the heart do not rely on complex equations and only contract when adjacent cells contract. As explained above, a set of transformations can be used to represent cell contraction when adjacent cells contract, thus modeling reality more accurately and improving the accuracy and speed of simulation. Can be made to. Therefore, the transformation makes it possible to solve global differential equations and avoid the vast amount of computation and approximation associated with applying the finite element method.

The transformation blocks and the interactions between the transformation blocks and the form data structure represent the key components of this transformation processing unit architecture. All processing of data relies on transformations that allow data to be retrieved, modified and moved in a single command. A single transformation allows a large amount of data to be collated, calculated, and taken elsewhere. Since the conversion command naturally collects and processes data of the same type of data structure, this data can be easily processed in parallel on GPUs, multiple GPUs, multiple CPUs, and processing cores in a virtual machine configuration. .. In particular, transformations marked to be applied n times can be calculated in parallel in one step. The transformation can be sent to a queue that processes the calculation. Independent transformations can be sent to different queues, thus allowing for a higher degree of parallelization.

Since all heterogeneous data can be written as a form on a machine running this architecture in a virtual machine configuration, the data normally processed in a single thread depends on the embodiment of the conversion processing unit in this specification. It can be processed in parallel with the use of transformations. For example, while using this architecture in a virtual machine configuration, on a computer with a CPU and GPU, the data processed by the conversion using the conversion processing unit is 15 times more than the same data processed through the traditional architecture. It can be processed up to 100 times faster.

The transformation collects matches with the input site, and each match can be processed in parallel. At this time, each match (which can be composed of a plurality of input site components) can be used to calculate the effect and the output site (each containing a plurality of component components). Output site matches are then collected and modified at the same time. Further, if the transformations need to be applied in parallel n times, n identical transformations can be applied in parallel. In addition, several different transformations that can be applied in parallel are also applied at the same time.

Therefore, form data and conversion commands written in a single way can transfer as much processing as possible to the GPU, parallel device, or Zetane machine, thus allowing speeds up to 1000x faster. ..

This architecture enables modularization. Each model described as a form and transformation can be saved / exported as a module. The module can then be used and reused in other larger models, thus facilitating data exchange and coordination. Modules and models can be stored in memory and can be accessed for processing by other modules by using memory addresses instead of data copies. This can improve access and processing speed. The interconnection of modules through memory addresses can reduce storage space usage and machine energy consumption.

In one embodiment, the concept of any domain can be expressed as a form through transformation, and the form can be processed (in parallel or non-parallel as machine language) through transformation, thus streamlining design and computation. And can be accelerated.

Visual representation of forms and transformations provides insight into the data and facilitates the construction and design of advanced models. The efficient representation of transformations and data allows users to interact more deeply and intuitively with computations and data. In particular, being "human-in-the-participatory" can enable significant optimizations that could not have been discovered without visual representation of transformations and forms.

Brief Description of Drawings Further features and advantages of the invention will become apparent from the following detailed description, taken in combination with the accompanying drawings.

FIG. 3 is a block diagram showing a conversion processing unit having a conversion control unit and a conversion logic unit and which can be connected to an external memory according to one embodiment. FIG. 3 is a block diagram showing a conversion processing unit including a conversion control unit, a conversion logic unit, and an internal memory according to one embodiment. FIG. 3 is a block diagram showing a conversion logic unit including a plurality of conversion blocks according to one embodiment. FIG. 3 is a block diagram illustrating a conversion logic unit comprising a plurality of interconnected conversion blocks according to one embodiment. FIG. 6 is a block diagram showing a conversion block including an input site block, an effect block and an output site block according to one embodiment. FIG. 3 is a block diagram showing a conversion block including an input site block, an input site range block, an effect block, an output site range block, and an output site block according to one embodiment. It is a figure which shows the input processing by the conversion logic unit which includes a plurality of interconnected conversion blocks according to one embodiment. It is a figure which shows the input processing by the conversion logic unit which includes a plurality of interconnected conversion blocks according to one embodiment. It is a figure which shows the input processing by the conversion logic unit which includes a plurality of interconnected conversion blocks according to one embodiment. It is a figure which shows the input processing by the conversion logic unit which includes a plurality of interconnected conversion blocks according to one embodiment. It is a figure which shows the input processing by the conversion logic unit which includes a plurality of interconnected conversion blocks according to one embodiment. It is a flowchart which shows the method for performing the conversion by one Embodiment. FIG. 6 is a block diagram of a system for performing the method of FIG. 12 according to one embodiment. It is a figure which shows the transformation including the input site, the input range, the effect, the output site, the output range and the conversion data by one embodiment. FIG. 3 shows a rectangular box following a triangle oriented in a 3D environment, according to one embodiment. FIG. 3 shows a rectangular box following a triangle oriented in a 3D environment, according to one embodiment. FIG. 6 illustrates an exemplary graphic representation of a given transformation with a feed in the relevant transformation data. It is a figure which shows the exemplary transformation for making three points. It is a figure which shows the exemplary transformation for making a solid line. It is a figure which shows the example of the generalization of a plurality of applications of a transformation. It is a figure which shows an exemplary transformation given a plurality of variables existing in a feed. FIG. 6 illustrates an exemplary transformation designed to stack points on top of each other. FIG. 3 illustrates an exemplary transformation designed to create a solid square. FIG. 6 illustrates an exemplary transformation designed to create a solid triangle. It is a diagram showing an exemplary transformation designed to create an accelerating rocket. FIG. 6 illustrates an exemplary transformation designed for rocket relative displacement. FIG. 6 illustrates an exemplary transformation designed to move a rocket seven times. FIG. 6 illustrates an exemplary transformation designed to move a rocket seven times using interpolation. FIG. 5 illustrates an exemplary customization of how the interpolation is created. It is a figure which shows the exemplary continuous transformation about an absolute position. It is a figure which shows the exemplary continuous transformation about a relative position. FIG. 3 illustrates an exemplary transformation for creating a continuous relative displacement of a rocket. It is a figure which shows the exemplary transformation for making the vector of n elements. It is a figure which shows the exemplary transformation for making a 3x3 matrix. It is a figure which shows the transformation including the two-part effect by 1st example. It is a figure which shows the transformation including the two-part effect by the 2nd example. It is a figure which shows an exemplary transformation including a feed and a two-part effect. It is a figure which shows an exemplary discrete transform. It is a figure which shows the addition of the tetrahedron to the face of the tetrahedron by one example. It is a figure which shows the addition of the tetrahedron to the face of the tetrahedron by one example. It is a figure which shows the addition of the tetrahedron to the face of the tetrahedron by one example. FIG. 6 is a diagram illustrating a transformation designed to replace a line segment with a triangle and its application, according to one example. FIG. 6 is a diagram illustrating a transformation designed to replace a line segment with a triangle and its application, according to one example. FIG. 6 is a diagram illustrating a transformation designed to replace a line segment with a triangle and its application, according to one example. FIG. 2 is a diagram showing the application of a transformation designed to replace a line segment with a triangle according to the second example. FIG. 2 is a diagram showing the application of a transformation designed to replace a line segment with a triangle according to the second example. It is a figure which shows the universe of the transformation which represents two interconnected neurons by one Embodiment. FIG. 44 shows a first exemplary transformation applied within the universe shown in FIG. FIG. 44 shows a second exemplary transformation applied within the universe shown in FIG. FIG. 44 shows a third exemplary transformation applied within the universe shown in FIG. FIG. 44 shows a third exemplary transformation applied within the universe shown in FIG. FIG. 44 shows a third exemplary transformation applied within the universe shown in FIG. FIG. 44 shows a fourth exemplary transformation applied within the universe shown in FIG. It is a figure which shows an exemplary model of a synaptic cleft. FIG. 51 shows exemplary transformations for modeling the release of 10 neurotransmitters in the synaptic cleft of FIG. 51. It is a figure which shows the model of the synaptic gap of FIG. 51 given the rectangular box which visually represents the output range by one Embodiment. It is a figure which shows the release of a neurotransmitter in a synaptic cleft and their absorption by a neuroreceptor. It is a figure which shows the release of a neurotransmitter in a synaptic cleft and their absorption by a neuroreceptor. It is a figure which shows the release of a neurotransmitter in a synaptic cleft and their absorption by a neuroreceptor. It is a figure which shows the release of a neurotransmitter in a synaptic cleft and their absorption by a neuroreceptor. It is a figure which shows the transformation for making the graphene sheet by one Embodiment. It is a figure which shows the result of application of the transformation for making a graphene sheet by one Embodiment. It is a figure which shows the use of the transformation for making a CO ₂ collector by one embodiment. It is a figure which shows the use of the transformation for making a CO ₂ collector by one embodiment. It is a figure which shows the use of the transformation for making a CO ₂ collector by one embodiment. It is a figure which shows the manual rotation of a satellite by one Embodiment. It is a figure which shows the manual rotation of a satellite by one Embodiment. It is a figure which shows the exemplary automatic signal finder conversion. It is a figure which shows the exemplary keyboard operation rotation of a satellite. It is a figure which shows the exemplary rotation of a satellite carried out through a button. It is a figure which shows the 1st exemplary transformation for moving a car. It is a figure which shows the 2nd exemplary transformation for moving a car. It is a figure which shows the making of the mileage meter by one Embodiment. It is a figure which shows the making of the mileage meter by one Embodiment. FIG. 6 shows one of two different visual representations of the same transformation, according to one embodiment. FIG. 6 shows one of two different visual representations of the same transformation, according to one embodiment. It is a figure which shows the graphical user interface for making a transformation by one Embodiment. FIG. 6 illustrates the use of the graphical user interface of FIG. 74 to create a tetrahedron according to one embodiment. FIG. 6 illustrates the use of the graphical user interface of FIG. 74 to create a tetrahedron according to one embodiment. FIG. 6 illustrates the use of the graphical user interface of FIG. 74 to create a tetrahedron according to one embodiment. FIG. 6 illustrates the use of the graphical user interface of FIG. 74 to create a tetrahedron according to one embodiment. FIG. 6 illustrates the use of the graphical user interface of FIG. 74 to create a tetrahedron according to one embodiment. FIG. 5 shows a tetrahedron obtained according to the steps shown in FIGS. 75-79 according to one embodiment. FIG. 5 shows a complex fractal obtained according to the steps shown in FIGS. 75-79 according to one embodiment. FIG. 5 illustrates the use of a given transformation with input sites, effects, and output sites according to one embodiment. FIG. 5 illustrates the use of a given transformation with input sites, effects, and output sites according to one embodiment. FIG. 5 illustrates the use of a given transformation with input sites, effects, and output sites according to one embodiment. It is a figure which shows the model obtained after applying the transformation shown in FIG. 83 continuously by one Embodiment. It is a figure which shows the model obtained after applying the transformation shown in FIG. 83 continuously by one Embodiment. It is a figure which shows the model obtained after applying the transformation shown in FIG. 83 continuously by one Embodiment. It is a figure which shows the model obtained after applying the transformation shown in FIG. 83 continuously by one Embodiment. It is a figure which shows the transformation which dynamically changes the input site match by one Embodiment. It is a figure which shows the exemplary transformation given a different range. It is a figure which shows the exemplary transformation given a different range. It is a figure which shows the tetrahedron obtained by applying the transformation of FIG. 91 by one Embodiment. It is a figure which shows the line segment by one Embodiment. It is a figure which shows the transformation for rotating the line segment of FIG. 93 by one Embodiment. FIG. 5 shows an input of the number of times a transformation is applied within a user graphical interface, according to one embodiment. It is a figure which shows the result of applying the conversion of FIG. 95 by one Embodiment. FIG. 6 illustrates an interface that presents a sequence of recorded transformations according to one embodiment.

Note that similar features are identified by similar reference numerals throughout the attached drawings.

Detailed Description An embodiment of the present invention is intended for a conversion processing unit for processing heterogeneous data. In one embodiment, the architecture allows for a unified universal processor implementation. There are several advantages to having the conversion processing unit interact directly with the inputs and outputs and store them directly in the conversion block. Registers are storage units with the fastest response times and can be accessed directly within the conversion block. This streamlines a very high performance computing pipeline where the input data is also inserted directly into the conversion block, which is processed at high speed by other conversion blocks, and finally the data is output almost directly to the output device. can do.

The following describes the computer architecture for the conversion processing unit that enables visual programming as well as high performance computing and simulation. Computer architecture represents a processing unit device suitable for processing heterogeneous data at high speed.

Using this conversion processing unit device, the user can apply an effect to a form existing in the universe. A transformation is created to apply the effect to the form. The conversion includes at least the effect applied to the form, the output site that defines where the effect should be applied, and optionally the conversion data. Alternatively, the conversion includes at least the effect, the input site that defines where the effect can get the information, and optionally the conversion data. A processing unit device having an architecture adapted to perform a conversion, as described below, is referred to as a conversion processing unit device.

FIG. 1 shows one embodiment of a conversion processing unit device 10 comprising a conversion processing unit 12 and a form conversion instruction set 14. The conversion processing unit device 10 communicates with an external memory or storage unit 20 for storing data, statements and / or instructions. It should be understood that the conversion processing unit 10 also includes an input and output mechanism for receiving and transmitting data.

The conversion processing unit 12 includes a conversion control unit 16 and a conversion logic unit 18 connected to each other.

The conversion control unit 16 is configured to coordinate the sequence of data movement to and from and between the components and subcomponents of the conversion processing unit device 10.

The conversion logic unit 18 includes instructions of a computer program by performing basic arithmetic operations, logical operations, control operations, and input / output (I / O) operations specified by the instructions.

The transformation logic unit 18 is configured to store all information about the transformation. In one embodiment, the conversion logic unit 18 is a control electronic circuit (ie,) that directs instructions related to the conversion by performing basic arithmetic operations, logical operations, control operations, and input / output operations specified by the instructions. It is equipped with a control unit).

The form conversion instruction set 14 contains all the stored conversion-related data and can be accessed by the conversion processing unit 12. The form conversion instruction set 14 includes instructions for data processing operations, conversion operations, control flow operations, and complex operations. Some of the operations described are collation, comparison, transform apply, replace, get, copy, superimpose, add, name, parallel and serial sequence of transforms, sequence and number of transforms, feed and interpolation. The form conversion instruction does not need to be separated from the conversion processing unit 12, and can be encoded in a reserved conversion block, a dedicated register, a storage unit, a memory or a disk. Furthermore, the form conversion instruction can also be constructed from an electronic circuit.

Although the conversion processing unit device 10 can be connected to the external memory 20, FIG. 2 shows one embodiment of the conversion processing unit device 30 including the internal memory 32. The conversion processing unit device 30 includes a conversion processing unit 34 and a form conversion instruction set 14. The conversion processing unit 34 includes a conversion control unit 16, a conversion logic unit 18, and an internal memory or storage unit 32 for storing data, statements and / or instructions.

As described below, the architecture of the transformation logic unit 18 can change as long as it contains all the information about the transformation.

FIG. 3 shows one embodiment of the architecture of the conversion logic unit 40. The transformation logic unit 40 has a plurality of transformation blocks 42a, 42b, each for storing all the information related to each transformation. .. .. 42n is provided.

In the illustrated embodiment, the conversion blocks 42a, 42b ,. .. .. 42n is the effect blocks 44a, 44b ,. .. .. 44n and their respective output sites or output site blocks 46a, 46b ,. .. .. Includes 46n. Each effect block 44a, 44b ,. .. .. 44n is the effect blocks 44a, 44b ,. .. .. Each output site block 46a, 46b, etc., includes a plurality of memory units or registers 48 for storing data for each effect associated with 44n. .. .. 46n is the output site blocks 46a, 46b ,. .. .. Includes a plurality of memory units or registers 48 for storing data associated with each output site associated with 46n. It should be appreciated that the memory unit 48 is configured to store data and can be defined by a plurality of stored bits.

In one embodiment, some conversion blocks 42a, 42b ,. .. .. Only 42n has some other conversion blocks 42a, 42b ,. .. .. It can communicate only with 42n. In another embodiment, some conversion blocks 42a, 42b ,. .. .. 42n are independent of each other and are any other conversion blocks 42a, 42b ,. .. .. It may not communicate with 42n. In a further embodiment, as shown in FIG. 4, conversion blocks 42a, 42b ,. .. .. All 42n are interconnected and, as a result, any conversion blocks 42a, 42b ,. .. .. 42n is any other conversion block 42a, 42b ,. .. .. It can communicate with 42n.

FIG. 4 shows a plurality of conversion blocks 52a, 52b, 52c, 52d, each containing an effect block and an output site block. .. .. The conversion logic unit 50 including 52m and 52n is shown. Each conversion block 52a, 52b, 52c, 52d ,. .. .. 52m, 52n are all conversion blocks 52a, 52b, 52c, 52d ,. .. .. Directly connected to 52m, 52n, and as a result, any conversion blocks 52a, 52b, 52c, 52d ,. .. .. The outputs of 52m, 52n are any other conversion blocks 52a, 52b, 52c, 52d ,. .. .. Inputs of 52 m and 52 n can be used.

In one embodiment, the conversion block may further comprise at least one control unit, as shown in FIG.

FIG. 5 shows one embodiment of a conversion block 60 that includes an input site or input site block 62, an effect block 64, an output site or output site block 66, and a control unit 68 that directs operation based on instructions. The input site block 62 includes a plurality of memory units 70 and a plurality of logical units 72. The effect block 64 and the output site block 66 have the same architecture as the input site block 62, the effect block 64 includes a plurality of memory units 74 and a plurality of logical units 76, and the output site block 66 includes a plurality of memories. Includes unit 78 and a plurality of logical units 80. The logical units 72, 76, and 80 may be arithmetic logical operation units, conversion logical units, references to other parts of the conversion logic units, and the like.

The conversion block 60 includes both the input site block 62 and the output site block 66, but the input site block 62 may be omitted, and as a result, the conversion block 60 may include only the effect block 64 and the output site block 66. Please understand that. Similarly, the output site block 66 may be omitted, and as a result, the conversion block 60 may include only the effect block 64 and the input site block 62.

In one embodiment, the control unit 68 may be omitted. Logical units 72, 76 and 80 may be omitted in the conversion block of the same or another embodiment.

In the illustrated embodiment, the logical units 72, 76, and 80 are included in the input site block 62, the effect block 64, and the output site block 66, respectively, but the logical units 72, 76, and 80 are converted into the conversion block 60. It will be appreciated by those skilled in the art that it can be located outside the input site block 62, effect block 64 and output site block 66 within.

At least one of the input site block 62, the effect block 64 and the output site block 66 may not include a logical unit, while at least one of the input site block 62, the effect block 64 and the output site block 66 contains a logical unit. Please also understand that it includes.

FIG. 6 indicates an operation based on an input site or an input site range block 102, an input site or an input site block 104, an effect block 106, an output site or an output site range block 108, an output site or an output site block 108, and an instruction. An embodiment of a conversion block 100 including a control unit 112 and a memory 114 is shown. The input site range block 102 includes a plurality of memory units 120 and a plurality of logical units 122. The input site block 104 includes a plurality of memory units 124 and a plurality of logical units 126. The effect block 106 includes a plurality of memory units 128 and a plurality of logical units 130. The output site range block 108 includes a plurality of memory units 132 and a plurality of logical units 134. The output site block 110 includes a plurality of memory units 136 and a plurality of logical units 138. The memory 114 includes a plurality of storage units 140. It should be understood that the transformation block 100 is suitable for performing transformations including input sites, input ranges, effects, and output sites, and output ranges.

The logical units 122 , 126, 130, 134, and 138 may be arithmetic logical operation units, conversion logical units, references to other parts of the conversion logic units, and the like.

Referring again to FIGS. 1 and 2, the form conversion instruction set 14 stores conversion data such as operations and complex instructions. For example, the form conversion instruction set 14 enables data processing and memory operations such as read, write and copy or data coding of data and addresses. The form conversion instruction set 14 can further enable operations for conversions such as collation, comparison, conversion application, replacement, acquisition, copying, superposition, addition, and naming. The form conversion instruction set 14 also enables control flow operations such as parallel and serial conversion application, conversion application ordering, application count control, conversion prioritization, and timeline creation. The form conversion instruction set 14 is also responsible for complex instructions such as feeds, interpolations, and conversion lifetimes (which give the time or order at which the conversion begins and ends).

In an embodiment in which the conversion block includes a memory unit such as a register or a memory unit 48, the conversion processing unit device may not include the memory unit outside the conversion logic unit. For example, the memory unit 32 shown in FIG. 2 may be omitted. In such an embodiment, the conversion block itself can be used to store data such as all data related to the input and output universes, which can be used for high speed access memory and storage. can. Using this technique, the conversion logic unit can be made up of only conversion blocks that can use the conversion block as a storage unit.

As mentioned above, the data can be stored in registers, external memory, internal memory, etc., and the data can be formatted and encoded in various ways. In one embodiment, the data is encoded as a form and transformation optimized for the transformation processing unit device 10. Forms can be used to represent any type of data and can represent a highly hierarchical structure. This is, for example, the body containing the liver and heart, where the heart is composed of cardiomyocytes and the cells themselves contain DNA composed of molecules composed of atoms. A form can be made to accurately represent such a hierarchical structure, along with data that allows the transformation to process the form efficiently.

In an embodiment where the form conversion processing unit does not include memory, data such as forms are stored directly in the registers of the conversion block. In this case, some conversion blocks can be used for storage and can have empty input and output site blocks, while effect blocks can contain data such as forms. Alternatively, depending on the access speed, the input site, output site and effect block can each contain a copy of the form.

In one embodiment, keyboards, mice, and other input devices can be connected to reserved or pre-assigned conversion blocks. Outputs such as monitors, VR headsets and other devices can also be connected to multiple conversion blocks, and the identification numbers or addresses of the conversion blocks.

In one embodiment, the control unit or part of the control unit may consist of a reserved set of conversion blocks. This allows the processor's internal processes to be modified for different types of optimization.

The conversion processing unit device described above performs the conversion, i.e., according to the external site, uses the conversion data and is optionally included in the input site, input range, and output range, as described in detail below. Please understand that the information is configured to apply the effect to the selected form.

7 to 11 show a plurality of conversion blocks 152a, 152b, 152c, 152d ,. .. .. An exemplary process for calculating and processing data using a conversion processing unit device 150 containing only 152m, 152n is shown. Each conversion block 152a, 152b, 152c, 152d ,. .. .. The control units 154a, 154b, 154c, and 154d, respectively, are attached to 152m and 152n. .. .. 154m, 154n, each effect block 156a, 156b, 156c, 156d ,. .. .. 156m, 156n, and their respective output site blocks 158a, 158b, 158c, 158d ,. .. .. 158m and 158n are given. The architecture of the conversion processing unit device 150 is clockless, and as a result, the device 150 corresponds to an asynchronous processor device.

Conversion blocks 152a, 152b, 152c, 152d ,. .. .. The 152m and 152n are all interconnected, and as a result, the conversion blocks 152a, 152b, 152c, 152d ,. .. .. 152m, 152n are other conversion blocks 152a, 152b, 152c, 152d ,. .. .. Data can be exchanged with either 152m or 152n.

As shown in FIG. 7, the conversion processing unit device 150 is configured to receive an input signal and output an output signal. The input signal can indicate, for example, the pressing of a key on the keyboard.

As shown in FIG. 8, some conversion blocks such as conversion block 152a are empty, while other conversion blocks such as conversion blocks 152b, 152d, and 152m are predefined and provide information about some conversions. include.

In this example, the key 1 is pressed and the conversion block 152a receives a signal from the keyboard that the key 1 has been pressed. The control unit 154a of the conversion block 152a updates the effect block 156a and the output site block 158a according to the received input signal. In this example, as shown in FIG. 9, the control unit 154a updates one of the registers of the effect block 156a with a binary number 1 , and represents one of the registers of the output site block 158a with the character K. Update by binary number.

The control unit 154a of the conversion block 152a converts the signal encoding the value of the effect and the output site into another conversion block 152b ,. .. .. By sending to 152n, the collation protocol is started. Upon receiving a signal from the conversion block 152a, the conversion block 152b ,. .. .. , 152n control unit 154b ,. .. .. 154n starts comparing the incoming signal of the output site with its stored value, and if there is a match, the stored value is changed to the incoming effect value. If there is no match, the stored value is neither replaced nor modified. In this example, the value included in the effect block 156d of the conversion block 152d corresponds to the value included in the output site block 158a of the conversion block 152a. Therefore, the value contained in the effect block 156d is changed to 1 as shown in FIG. At the digital circuit level, matching can be achieved using a digital comparator consisting of multiple logic gates, which can be achieved directly without a normal control unit. The logic gate corresponds to the control unit. In addition, data transmission, data reception, and instruction and operation initiation can be performed directly at the conversion block level without the use of control units.

Next, the control unit 154d of the matching conversion block 152d sends a signal indicating the new effect value to the other conversion blocks 152a, 152b, 152c, 152n, 152m, and the other conversion blocks are each of them. Attempts to match the output site values of (ie, those output site values stored in those output site blocks 158a, 158b, 158c, 158n, 158m) with the new effect values of the conversion block 152d. In this example, there is a match for effect 2 (or 10 in binary form) and the conversion block 152m associated with the conversion by output site 1. Next, the conversion block 152m sends its effect value and its output site value to the conversion block 152d, and replaces the recently updated value 1 with 2. At this time, the conversion related to the conversion block 152d corresponds to the effect having the values 2 and O at the output site. For example, the newly updated conversion 152d can then signal a plurality of conversions responsible for updating the monitor pixel values to indicate the number 2. In another example, the newly updated conversion 152d can also trigger or participate in additional internal processor computations.

In the above example, only one match was possible in each matching attempt from the transformation, but it should be understood that multiple matches may occur in each matching attempt. If there are multiple matches, each matched transformation is updated and the computational process, such as matching other multiple transformations data, continues. This not only enables the architecture of the conversion processing unit device to be clockless, but also enables a high degree of parallelization in nature. In addition, the number of matching instances can increase exponentially (each of the five matches triggers five matches that can match the other five transformations, thus 5,5 * 5). 5 * 5 * 5 simultaneous conversion events are given).

In one embodiment, the conversion block can also include an input site range (for storing the input range) and / or an output site range (for storing the output range). In this case, referring back to the above example, the matching protocol does not send the matching signal to all the conversion blocks that it attempts to match, but only to a subset of all the conversion blocks. For example, the signal from the conversion block 152a given the effect 1 and the output site K may be transmitted only to the conversion blocks 152i-152k. This can be done if the output site range or input site range of the conversion block 152a includes a list of registers / conversion block identification information (or address of the register) to which the matching signal will be transmitted. With such a configuration, the computer architecture (or the user defining the transformation block) can reduce matching time or energy consumption by limiting the "range" of the match search to a predefined set of transformation blocks. Can be done.

In addition, the input site range and / or the output site range can provide a way to distinguish similar conversion blocks. In the above example, there may be multiple conversion blocks that match the conversion with effect 1 and output site K. There may be conversions with [Effect 1 and Output Site O] and [Effect K and Output Site M], but depending on the output site range, one of them may match, for example, [Effect 1 and Output Site O]. It is possible to limit to only one. This is because the identification information is included in the output site range.

The transmitted signal or matching request is not only ensured to be transmitted for output site matching, but can also start with other conversions. For example, a conversion block with an input site can initiate (or receive) a signal. Input site matching works like an output site by allowing input site matching against the registers listed in the input site range.

In one embodiment, the conversion block is data (but not limited) that can be read and written by the control unit (s), the conversion itself, any number of other conversions or possibly any external device. It can also have a data section containing a register with a number of forms or references to the register). Examples of such data can include the number of times a conversion block initiates a matching protocol, or the number of collations that can be performed by this conversion block.

The above architecture of the conversion processing unit device can also be reproduced on a virtual machine. One with a CPU®, OpenCL®, OpenMP®, OpenCV®, Complete Shaders®, multithreaded libraries, parallel computing libraries and / or other types of libraries. Alternatively, multiple GPUs, cores, CPUs, processing devices, computing devices and / or field programmable gate arrays (FPGAs) can be used to approximate all or part of the architecture on a virtual machine.

The results shown below were achieved using the above architecture implemented on a virtual machine with a CPU and GPU. The virtual machine thus obtained was used, among other things, to train a 3D artificial neural network, which is a system of neural networks, to simulate DNA replication, and to create cloud simulations.

The following describes how to apply the transformations made possible by the above architecture.

FIG. 12 shows one embodiment of Method 200 for visual programming. Method 200 can be performed by the conversion process design described above implemented in either hardware or a virtual machine.

In step 202, the conversion is received. In this case, the conversion includes at least the identification information of the input site, the output site, the effect and the identification information of the conversion data. However, as mentioned above, the conversion may include only effects and input sites, or input and output sites, and the conversion data is optional.

At step 204, the input site is retrieved from the input universe. The input site identifies at least one form from the input universe from which the information related to the transformation is retrieved. At step 206, the output site is retrieved from the output universe. In one embodiment, the output site is retrieved from the output universe using the information retrieved from the input site. The output site defines a subspace of the output universe (ie, a given form contained within the output universe) to which the effect associated with the transformation is applied.

In step 208, the effect defined in the transformation is applied to the retrieved output site using the transformed data and the information retrieved from the input site, thereby obtaining a modified output site.

In step 210, the modified output site is output. In one embodiment, the modified output site is stored in memory. In the same or another embodiment, the graphic display of the modified output site is displayed within the representation of the output universe, as described below. In yet another embodiment, the modified output site may be used as an input for performing a second conversion. For example, referring to FIGS. 8-11, the modified output site has its associated conversion blocks 152a, 152b, 152c, 152d ,. .. .. At least another conversion block 152a, 152b, 152c, 152d, from 152m, 152n. .. .. It can be transmitted to 152m and 152n.

In one embodiment, the input site can be used to modify any part of the transformation. For example, you can use input site data to instantiate, update, calculate, or modify any part of the output site, transformation data, output range, and effects.

In one embodiment, the conversion does not include the identification information of the input site and / or the conversion data. In this case, step 204 to retrieve the input site from the input universe is omitted, and / or the step to retrieve the transformed data is omitted, and the effect is applied to the output site without considering the information about the input site and / or the transformed data. To.

In another embodiment, the conversion does not include output site identification information and / or conversion data. In this case, the step 206 of retrieving the output site from the output universe is omitted, and / or the step of retrieving the transformed data is omitted, and the effect is applied without considering the information about the output site and / or the transformed data.

In embodiments where reception of output sites and conversion data is omitted, step 202 comprises receiving only the effect. After that, only the effects and input sites are received. Step 206 then resides in retrieving the received input site within the input universe, i.e. finding a match for the input universe of the received input universe and retrieving information related to the retrieved input site. .. Step 208 is to modify the effect using the retrieved information associated with the input match in order to obtain the modified effect. Next, the effect changed in step 210 is output. The modified effect may be stored and / or output for display in memory, such as in an effect block.

In one embodiment, step 204 further comprises retrieving the input range associated with the input site and / or retrieving the output range associated with the output site. In this case, the effect is applied using information related to the retrieved input range and / or information related to the retrieved output range.

In one embodiment, method 200 further comprises displaying a user graphical interface that includes a first section for representing the output universe and a second section for displaying a graphic representation of the transformation. In one embodiment, the second section contains a first window for defining input sites, a second window for defining effects, a third window for defining output sites, and transformation data. Includes a fourth window for entering. It should be understood that the user interface may change as long as the visual representation of the output universe and the visual representation of the transformation can be shown to the user on the display.

In one embodiment, the input site is displayed in the first window, the effect is displayed in the second window, while the output site is displayed in the third window.

As described in more detail below, the user can draw and edit the contents of any window.

The method 200 described above can be implemented as a computer program product comprising computer-readable memory for storing computer-executable instructions that perform the above steps when executed by a computer.

FIG. 13 shows one embodiment of the system 230 for visual programming. The system 230 includes at least one processing unit 232 such as a CPU and / or at least one GPU, a communication unit 234, and a memory 236 in which instructions and data are stored. The processing unit 232 is configured to perform the steps described above in Method 200. In one embodiment, the system 230 corresponds to a virtual machine configured to reproduce the conversion processing unit architecture described above.

As mentioned above, the conversion may consist of effects, input sites and / or output sites, and optionally conversion data, input ranges and / or output ranges. As a simple example, the output site can be thought of as the form changing within the output universe, and the effect shows the same form, but in a new state, for example, in a new position or with a new shape. It is in. You can think of a piece of paper (or an expression of a piece of paper) at the output site, and the effect is that this piece of paper is crumpled and looks like a ball. Here, the conversion effect corresponds to the action of crumpling the paper into a ball. This transformation can be applied 100 times to a stack of 100 sheets to obtain 100 crumpled paper balls.

If present, the input site is used to extract information from a subspace of the input universe, i.e., from at least one form selected within the input universe. The effect of the conversion is then based on the information of the selected form (s) and, optionally, the conversion data including, but not limited to, the number of times the conversion has been applied, the style and type of the conversion, and so on. Applies.

It should be understood that any suitable method for selecting an input or output site can be used. For example, copying selected forms from the system, using drag and drop features, importing other systems or parts of other systems, importing files from other software or devices, drawing tools. Insert an input or output form by creating a form directly using software features such as Transforms, Graphical User Interface Buttons, Graphical User Interface Buttons Connected to Transforms, Forms and Graphical User Interface Buttons Specified by Transforms. be able to.

In one embodiment, a copy of the form may be automatically displayed in other parts of the conversion when the form is automatically inserted into one part of the conversion or at the request of the user. In one embodiment, the automatically generated copy can be identical to the inserted form. In another embodiment, the automatically generated copy may differ from the inserted form. For example, the transparency of the auto-generated copies may be different. The user can then decide which element of the copy should be retained, deleted, replaced, edited, or modified. This enables conversion and rapid system construction. A system should be understood as a form, a transformation, a collection of forms, a collection of transformations, or a collection of forms and transformations.

FIG. 14 is an exemplary graphic representation 250 of a transformation that includes an input site 252, an input range 254, an effect 256, an output site 258, an output range 260, and conversion data 262. The input range allows you to select a portion of the input universe 264 where you can select an input site match, while the output range can apply effects to the output sites of the output universe 266 existing inside. It makes it possible to specify a part of the output range that can be done.

In one embodiment, the following commands / languages can be used to define the transformation.

The symbol ": =" or text entry section can be used to name a form, transformation, or universe.

"Copy / Take" means that the form selected in the input site, that is, the input universe, remains in the input universe after the effect is applied (copy) or is entered after the conversion is applied. It can be used to specify whether it disappears (cuts) from the universe.

"Add / Replace" means that the output site, that is, the form selected in the output universe, remains (adds) or is replaced in the output universe after the transformation is applied. Can be used to specify (replacement). In some cases, forms overlap each other, so it is permissible to have multiple copies of the same form in the same location.

"Number of times (#times)" can be used to indicate the number of times the conversion is applied.

"All" can be used to indicate that the transformation modifies each of the forms that satisfy the input and output sites of the transformation at that time or at the current position in the timeline once.

"Always (Always)" to indicate whether the transformation is always ready to be applied or can be applied an infinite number of times (in some cases, as long as the input and output conditions are met). Can be used for.

"Application #" can be used to indicate the order in which the various transformations are applied. If there are several transformations in the system, the one with the lowest number is the one that applies next.

"Lifepan parameters" (not shown) can be used to indicate within how many steps the transformation can be applied and how many steps the transformation can be applied. These parameters can refer to the global step counter and indicate the step (integer or interval) to which the transformation can be applied.

Other types of transformation data (not shown) include, but are not limited to, instructions on whether the transformation remains or disappears, the group to which the transformation belongs, and whether the transformation should be applied in parallel, sequentially, or simultaneously. , Transformation type, transformation style, transformation name, constraint, global axis or local axis, etc. may be included.

Conversion Activation and Process In one embodiment, if the conversion contains an input site and the input site has a match within the input universe, the conversion can be activated and performed. If there are inequalities or conditions that must be met in the input site, the conversion will not be applied until the inequalities are met, even if the output site has a match in the output universe.

In one embodiment, the process of performing the conversion is as follows.
First, a match between the form defined in the input site and the form in the input universe is searched. If a match is found, the identified form is selected in the input universe.

It is then determined, if present, whether the inequality, condition, or equation defined within the input site or input range is satisfied, based on the information associated with the selected form in the input universe. In one embodiment, the inequalities and equations to be satisfied can be included within the input or output range, if any.

Effects , output sites and transformation data are updated with the selected information by replacing the appropriate variables (or *) with information or modified information. In this case, the conversion is said to have been activated and the values are said to have been initialized.

The forms contained in the output universe that match the forms defined within the output site are then selected.

The transformation then replaces the selected form in the output universe with the form defined in the transformation's effect. In one embodiment, the form defined in the effect can be overlaid on the selected form in the output universe.

In one embodiment, if the input site is always connected to or matches a given form in the input universe, changes to a given form in the input universe trigger the execution of the above process. In this case, the conversion may not include any output site and therefore may not include any output range. An example of a form that is always connected corresponds to a scenario where there is one form in the input universe, the same form is defined in the input site, and as a result the transformation is set to copy instead of cut. In this case, the conversion is automatically performed when a change that exists in the input universe and is made to the form defined in the input site is detected.

Applying Transforms Transforms, when including input and output ranges, can be applied as follows. Input and output ranges can be the entire system, empty, form, form type, system subsystem, system multidimensional space, or multidimensional space around the system form (eg, volume around the edges of a 3D mesh). And so on. The input range provides information about where the transformation is allowed to extract the information. That is, the input range defines a subspace of the input universe from which the identified form can be selected within the input site. The input site defines a form that can be selected from the input range. The effect consists of a form that relies on the information of the selected form in the input range. The output range provides information about where the transformation can be applied in the output universe. The conversion is applied by selecting a form similar to or equal to the output site from the output range. The selected form of the system is then modified, added, added horizontally, added to a nearby hierarchical position, or replaced with the corresponding form from the effect. The correspondence between the form of the output site and the effect can be shown in various ways, for example by displaying a line between them or by using data displayed in different places.

Input sites and transformations given effects can be understood as taking forms from a collection of forms in the input universe and using them to build objects called "effects". The selected form is removed from the input universe or remains unchanged. Transformations with output sites and effects can be understood as selecting a form from the output universe and then adding, replacing, and modifying the selected form. The form at the output site can be present in the effect, especially if the element is added to the selected form. The transformation can have both an input form and an output form. Therefore, you can create an effect from the input and then apply it to modify the form from the system.

In one embodiment, the selected form can be the name or type tag of an object. The transformed data may include instructions as to whether the tag itself or the content of the tag should be selected. For example, this allows the tag "molecule" to be selected and the molecule associated with the tag "molecule" to be used or replaced. It can be extended to an object different from the tag or name, using a notation (symbol or color) that indicates that everything attached to a particular form needs to be selected or changed. In addition, it is possible to define the notation (symbol or color) selected to modify all forms that do not have the given structure identified by the notation. For example, a form with a tag molecule can be selected, but a transformation without a carbon atom (C) can be defined.

Transformed data The way the transform extracts, builds, applies, and behaves in the system can be tuned by the transformed data. Examples of such transformation data are instructions and specifics regarding the number of transformations applied (finite or infinite), the order in which the transformations are applied, and whether the transformations are removed or stayed when the transformations are no longer applicable. Instructions on whether the transformation should be applied to all objects of type, whether the transformation represents an interpolation step, how the transformation scales or rotates the object, whether the transformation is absolute or relative. Instructions on how it behaves, indicators of how the transformation affects the coordinate system, the name of the transformation, the name of the system in which the transformation is used, the names of the creators and editors of the transformation, and comments related to using the transformation. , Local and remote repositories, ratings, etc. The transformation data can be assigned to the entire transformation, but different datasets can be assigned to different collections of parts of the transformation. For example, data can be assigned to a couple of input sites and effects, or a couple of output sites and effects. It can be used, for example, to create effects based on multiple forms of a particular type, and then apply transformations to modify a single object.

In one embodiment, useful types of conversion data are referred to as "post" and "pre". Assigning a "post" to a transformation indicates that this transformation will be applied after each application of the non-post transformation. This can be used in 3D animations to allow the mesh of points to follow a particular form, as shown in FIGS. 15a and 15b. For example, a post transformation that adds a 3D mesh on top of a triangle can cause the 3D mesh (or rectangular box) to follow the position and orientation of the triangle. After moving the triangle, the ship is updated to the position following the triangle due to the post transformation.

In one embodiment, a form can be inserted as a subset of another form, and as a result, the interpretation of the subset's form depends on its parent form. For example, a 3D mesh can be inserted as a subset of triangles, and as the triangle moves, the mesh is interpreted based on the position of the triangle, so the mesh follows it.

Understand that you can build a sequence of simultaneous and series of transformation transformations, in which case the transformations will be applied sequentially so that the next transformation cannot be applied until the previous transformation still exists. I want to be. Since the sequence itself is also a form, you can assign a collection of data. In one embodiment, the conversion sequence number indicates which conversion should be applied next. Multiple simultaneous sequences without numbers or equal sequence numbers in the converted data can be applied simultaneously or sequentially, depending on their respective priorities. Alternatively, the order of execution is determined from the framework's random functions, based on the transformation data information, using the probabilities assigned to each transformation. Sequence numbers can be given by functions and operations. For example, the application can be made to alternate between two or more transformations. In one embodiment, the sequence can have the property of being active, and the transformations it contains apply in some or all timeline steps. In one embodiment, changes in the universe or events trigger the retrieval of the sequence processed by performing the transformations they contain.

Self- and higher-order transformations It should be understood that there may be no restrictions on where the transformation can extract information or where the transformation can be applied. For example, you can apply a transformation to modify any part of any other transformation, system data, or data associated with the transformation. The transformation can even extract information or modify a given one of its own parts. These features provide opportunities for higher-order systems and the incorporation of reproductions in the model.

Timeline access You can build a system timeline by recording the previous state in memory at each step or as the system changes. The timeline can be thought of as a history of the system. Similarly, a timeline for each element or form of the system can be recorded. For example, you can record where the edges of the mesh were and then make them accessible. In this case, some of the transformations, such as effects or output sites, can include not only the form, but also the previous state of the form recorded in the timeline. This can be useful for creating modeling related to differential calculations. For example, if an edge is in a specific position and then in a specific position in the past, you can build a transformation that removes the edge. It is possible to use the future of the timeline, but this requires running the system multiple times or running multiple paths. For example, if the edge is in a given actual position and will be in a particular position in the future, the edge may be removed at this point. In this case, this assumes that the future is determined after running the system once without removing the edges.

The following shows more accurate examples related to future timeline access. If a particular future form exists (or is in the required state) in the system in a future step, the transformation can be applied to the system in step _S0 . If such a transformation can be applied, the system will continue to run the system without applying the transformation and check if there are future forms in the system at the specified steps. If the form exists, the system returns to step _S0 , applies the transformation, and runs the system. If the form does not exist in a future step, the system will continue to run from there. This technique can be used to optimize the system or avoid adverse effects within the system.

Transformations that require access to the timeline can simultaneously access multiple past, present, and future steps in many different forms.

Hidden / Show, Private / Public, and Interface Each system, transformation or form can be shown or hidden and / or selected as public or private. This helps hide the underlying data that you don't need to see when you run the system. For example, a 3D mesh can only display faces, not vertices or edges. However, the transformation still applies to hidden forms, even if the edges remain exposed. Based on the framework's hidden / visible and private / public features, it is possible to create an interface where only some information is exposed and can be edited. For example, it can have a system composed of n equilateral triangles with sides of color c and length d. You can create an interface that looks like a triangle [n, c, d].
In this case, the triangle [6, green, 1/2] represents six green equilateral triangles with sides of 1/2 in length. This ability to create an interface can also be incorporated as a native feature of the system.

GUI Features of the Framework In one embodiment, the system has a graphical user interface (GUI) that includes buttons that can be created within the framework itself or added by using the framework itself. ) Can be included. You can apply the conversion to your system by clicking the GUI button, keyboard button, mouse button, or by starting typing. This allows the user to fully customize the user interface. The initial GUI may come with a framework, but the user can add buttons to start different conversions or run different systems. Interestingly, this allows users or developers to create 3D GUIs for virtual reality and augmented reality applications. The conversion processing unit and method can be used to create an interactive 3D website that can be navigated by software that operates in a manner similar to a web browser.

Incoming and Outgoing Data The system may or may not be connected in real time to other software or hardware with specialized forms called incoming and outgoing forms. This type of form may be public or private data accessible by another software or hardware. Here, outgoing data is taken as an example. After constructing a 3D mesh model of a robot moving in this system, the joint angle of the arm can be selected as the transmission form data. Real-world robot hardware can access outbound form data to update the servomotor angle. Similarly, data coming from various sensors on the user's body can be used to update the human virtual biological model, including incoming form data. The model can then have the same heart rate or respiratory rate as the user.

Since a form can be a collection of many forms, outgoing data can be used to service the system or system data after a request comes from an incoming data form. This can be a useful component for the "network of projects" described below.

In one embodiment, the framework is a 3D mesh in various formats, data from databases and tables, or languages such as HTML / CSS, C ++®, Java® and Python®. You can import various types of files such as code. This can be a native feature of the software or interpreter built as a system or form within a form, converting the framework to which the conversion is applied to a text or computer file for translation between languages.

Multiscale This conversion processing unit device can enable multiscale (spatial and temporal) navigation and processes. For example, in the heart model, cells can be clicked to expand the cell's internal system. It can also be progressively zoomed towards the cell, revealing the internal system by making the cell membrane transparent and revealing the internal model of the cell. When importing another system into the main system, you can specify various parameters of the system data related to the incorporation of this system. Examples of such data include size scales and time scales. For example, when importing cardiomyocytes into the heart system, it is necessary to specify the cell size and time scale for the heart. Intracellular transformations will be applied much more often than transformations at the heart level. The heartbeat is on the second scale, while the intracellular transformation is on the nanosecond scale.

In one embodiment, each system or form has a tick number (integer, floating point number, double, rational or real number) associated with it, and the tick number can be updated and changed. Tick numbers help indicate which transformations in the system can be applied next. It can be used to adjust the speed and time of passing through systems and subsystems.

Interpolation At least one intermediate transformation can be added between the two applications of the interpolation. This is called interpolation between the two transformations. Also, the conversion can be replaced with a plurality of conversions. This is called transformation interpolation. This can be done manually with the system by inserting a sequence of transformations or by using the system's native interpolation functions. For example, a transformation that moves a point by one unit in the z direction can be changed to a sequence of transformations using an interpolation function such that each transformation in the sequence moves the point by 0.01 units in the z direction. This can improve the accuracy of the system.

When performing transformation interpolation or interpolation between two transformations, the number of transformations used to generate the interpolation can be finite. Alternatively, this number may be infinite or continuous. This allows the introduction and use of abstract concepts rather than continuous transformations within the framework. This relates to the classical mathematical concepts of solid lines, real numbers, continuity and continuous functions. Interpolation can be achieved by using the feed described below.

Conversion Feeds The conversion feeds will be described below with reference to FIGS. 16-31. The conversion feed is part of the conversion data associated with the conversion.

A feed consists of variables and their associated ordered form sets. The feed can be understood as a list of numbers (or objects) that are sequentially pushed into the transformation to create instances of this transformation to be applied. The related set can be any type of set, such as a finite set of numbers, an infinite number of sets, a real interval, a set of forms, a set of transformations, a sequence of objects, a set of symbols, data, and so on.

An example of this set is given below. [0,1], [0, infinity]; [ 2,5]; [n] = {1,2,3,4. .. .. }; [0,1] U [4,12]; M × N; {abd, a, cc, c, e, t,%}; {1,2,3,4,5. .. .. }; {1,2,3,4.5,12.2, -15.2}; {1,2,2,2,2,1,1,10,0.5, e}; real numbers, complex numbers , A set of rational numbers and / or integers; {conversion 1, form 1, form 2, a, 4.3, 6, pi}; etc.

In one embodiment, the feed consists of multiple variables and their respective sets. There can be an order in which the variables are used first. There can also be feed constraints that variables must meet. In some embodiments, the constraint appears at the conversion input site.

In some embodiments, the feed allows conversion to:
-Extending the transformation to be continuous as well as discrete,
-Creating a continuous shape,
-Interpolate between discrete transforms so that the interpolation is completely controlled,
-Accessing the concept of continuity and real numbers of intensities,
-Performing differential calculations,
-Generalizing the concept of the number of times a transformation is applied,
-Used as a for loop or while loop,
-Representing vectors and multidimensional arrays, and even tensors,
-To behave like a loop to instantiate multiple transformations, similar to a loop for instantiating multiple objects in an object-oriented language, and / or-to transform using a feed.

Feed in operation In one embodiment, the transformation can consist of input sites, effects, and output sites, as well as a feed of one variable with each ordered set. Variables can appear anywhere in the transformation and any number of times (zero, finite, or infinite), including in the data of the transformation, including input sites, output sites, effects, and even the feed itself.

To apply the transformation, the instances of variables contained in the transformation are replaced with the first element of the ordered set. When the transformation is applied, the variable is replaced by the second variable in the set, then the second transformation is applied, and the process has all the transformations for each element of the set of variables. Continues until applied once.

FIG. 17 shows a first example in which the transformation is applied three times for values 1, 2, and 3. The variable t is replaced by 1 and the transformation is applied, thus adding a white dot at a distance of 1 unit from the black dot. T is then replaced by 2 and the transformation is applied, thus adding a white dot at a distance of 2 units from the black dot. Finally, t is replaced by 3 and the transformation is applied, thus adding a white dot at a distance of 3 units from the black dot.

FIG. 18 shows an example of a transformation configured to create a solid line. Similar to the example of FIG. 17, a line of length 2 is created by sequentially feeding all the elements of the interval [0,2] to the transformation.

It should be understood that only a finite number of points are displayed in the display unit. However, hidden real points can be accessed as needed, such as when it is necessary to determine the intersection of two solid lines or the collision point between objects.

FIG. 19 shows a generalization of the number of applications. In this example, the variable t does not appear in the transformation. In practice, instead of defining a feed, you may write 100 times. Note that the type of symbol in the set is not important as long as the variable t does not occur in the transformation and there are 100 elements in the related set. In particular, a set containing the symbol "@" 100 times acts as a 100-time application of the transformation.

FIG. 20 shows an example of a plurality of variables in a feed. Having two variables in the feed makes it possible to generate a grid of elements. In one embodiment, the number of variables in the feed is unlimited and multidimensional arrays, matrices, and objects similar to tensor coefficients can be created. It should be understood that in this example, t = 1 is selected, then h = 1. After that, h = 2 is selected with t = 1, and then h with t = 1.
= 3 is selected. The process is then restarted by acquiring t = 1 and h = 1 until all possible combinations have been acquired. It is necessary to understand that the same process is executed for more than two variables. In one embodiment, the order in which the elements are selected can be shown in detail in the transformation data.

FIG. 21 shows an example in which a plurality of points are stacked on each other.
FIG. 22 shows an exemplary transformation involving a continuous set for the feed to create a plain or continuous square, while FIG. 23 shows the feed for creating a plain or continuous triangle. An exemplary transformation involving a continuous set of is shown.

In the case of a triangle description, the conditions at the conversion's input site must be met before the conversion is applied. This constraint indicates that only points under y = -x + 1 are added to the universe.

FIG. 24 shows an exemplary transformation for accelerating a rocket. Rocket acceleration depends on the number in the feed. It should be understood that 5t can also be selected to be any type of function f (t).

FIG. 25 shows an exemplary relative displacement of a rocket. In this example, the relative displacement of the rocket is defined based on the feed value 1/7 and the function 2t.

In FIG. 26, the transformation moves the rocket by 14 units, but requires the rocket to interpolate in 7 steps. The interpolation can be represented by the feed shown in FIG.

FIG. 28 shows that the mode in which the interpolation is performed can be customized to reflect behavior such as acceleration.

FIG. 29 shows an exemplary continuous transformation of absolute position in which another rocket is inserted into a new position with respect to the global axis when the rocket is removed.

FIG. 30 shows an exemplary continuous transformation for relative position. In this example, the transformations are applied relatively. You can access multiple values at once within a set. In this case, t1 = 0, t2 = 0.25, t3 = 5, t4 = 0.75 and t5 = 1. This makes it possible to calculate the difference and generate a displacement of one unit increment.

As shown in FIG. 31, continuous relative displacement can be performed using dt instead of delta t.

FIG. 32 shows an exemplary transformation for creating a vector of n elements. For example, as shown in FIG. 33, a 3 × 3 matrix can be created.

You can add variables to your system by adding a resolution variable symbol or by indicating that the form is a variable. Usually, when a variable is resolved, it needs to indicate what the system wants to be or what it wants to do. This is similar to solving for x in a classical algebraic equation with the equal sign "=". For example, taking cancer cells that continue to replicate can obtain the change of variables T applied to the cancer cells and indicate that the system is desired to be a cell that no longer replicates. This system is then used to solve this for T. Solving about T by using a resolution system created within the framework, using incoming and outgoing data forms, using external solutions, or manually entering possible transformations. You can also. The result is a set of various possible forms of T that stop the replication of cancer cells. Multiple types of mathematical and programming techniques can be modeled and implemented directly in the framework, allowing multiple powerful tools to be used simultaneously to solve difficult systems. Examples of tools that can be implemented in the framework include artificial neural networks, equation solving, mathematical optimization, and various machine learning algorithms.

Networking a project When you create a project or system, the system is in the repository and can be public or private. A new project P can be started by describing what needs to be achieved on a web platform, private local server, or peer-to-peer based network. The user can submit the system for the project or improve the system submitted by others in this project. The user can vote for the best system, that is, the best system that most closely matches the requirements of the project.

Other project systems can be fully imported into the system (or project) and adapted to it. For example, instead of importing the entire System Sub into System Sys, information can be sent from Sys to Sub and asked to have Sub calculate the return value of Sys. This enables distributed computing and reduces project and system duplication. Some projects can be private or public. This makes it possible to create a large network of projects where a large number of users can develop and create large and useful projects. Each of these projects can consist of multiple other projects that themselves depend on other projects. Each included project can be thought of as a library of script-based programming languages such as C ++.

In one embodiment, the effect defined within the transformation can include multiple parts to aid expression. For example, an effect can contain two parts. If an input site is selected and its value is used within the effect and output site, the output site form selected by the output site will be modified by the form in the output portion of the effect and selected by the input site. The form is modified by the form in the input part of the effect.

FIG. 34 shows an exemplary transformation with a two-part effect. Here, the value 12 of the initial universe is imported into the conversion, resulting in K = 12. Next, K = 0 is replaced with K = 12, and H = 12 is replaced with H = 1.

FIG. 35 shows a further example of a transformation with a two-part effect. In this example, the values 12 and 8 from the initial universe are imported into the conversion, resulting in M = 12 + 8 = 20. Next, H = 12 is replaced with H = 0, K = 8 is replaced with K = -8, and M = 0 is replaced with M = 20. Next, K = 0 is replaced with K = 12, and H = 12 is replaced with H = 1.

FIG. 36 shows an exemplary transformation including a feed and a two-part effect.
It should be understood that feeds can be emulated for discrete feeds using only the components of the tripartic transformation without feeds. The discrete transform shown in FIG. 37 obtains the value of t from the input universe containing the ordered elements 1, 2, and 3. After applying the transformation, the element on the left is shifted to the right. This makes it possible to select the second element of the ordered set when applying the next transformation. If the feed is a real interval or an infinite set, a new procedure needs to be defined to extract the elements from the continuous ordered set. In one embodiment, a feed definition can be added to the system, which is very suitable for processing intervals and has many other uses.

Further examples of conversion will be described below.
In the first example, to create a transformation, the user selects a tetrahedral face present in the output universe (see Figure 38a) and sends or drags and drops it to the transformation's output site (Figure 38b). See). Automatically, triangular faces will appear at the effects and output sites. The user can then edit the effect by adding a mesh on the face or shrinking the triangle. If the user adds a smaller tetrahedron to the surface (see FIG. 38b) and adjusts the conversion data by setting the number of times to 3, the conversion will be applied. When the system is run, smaller tetrahedra are added one after another on the faces until there are smaller tetrahedra on the three faces (see Figure 38c).

The user can also create another transformation in which a small cube is added to the triangle. When two transformations are placed on the tetrahedron of the system at the same time as another transformation and the system is run, only one of the two is applied. In this case, the user can determine that the likelihood of applying the cubic transformation is 80% and the likelihood of applying another transformation is 20%.

In another example, the user imports the molecule into the system, selects it, and sends it to the output site. The user can also add D in the input site and add it to the effect. Moving the effect numerator to the right opens up a place where you can insert a value or function. Drag and drop D from the effect, or type D into that space and it will connect to D at the input site. The user then applies the function to D by writing 1 / D. Here, when the transformation is applied to the molecule, the displacement of the molecule depends on the value of D from another location in the system or even from the incoming data. Before the transformation is applied, the value of D is read to determine the displacement of the molecule. In this case, when D = 100, if this transformation is applied to the molecule 7 times, the molecule will be moved 7 times to the left by 1/100 unit.

All previous positions of the molecule can be stored in the timeline so that the transformation can be accessed using the timeline. For example, to measure the average velocity from 5 steps before and from the present, the user writes or writes the global position of the current molecule (P0) and the global position of the molecule of the form 5 steps before (P5) in the input site. You can choose. The user writes S = (P0-P5) / 5 in the effect. Here, the average velocity is calculated each time this transformation is applied to the molecule. This transformation is left in the corner of the system and can be applied every time the system is modified. By hiding the input site for that conversion, only the velocity data will be displayed in the corner.

FIG. 39 shows an exemplary conversion including input sites, effects, and output sites (conversion data omitted). The application of the conversion is carried out as follows. First, the form in the input universe is selected. The selected form matches the input site, where the star (*) can be anything as long as the forms around the star match.

In this example, A = 3 is selected because "A =" is the same and any star symbol can be selected, and therefore the number 3 is selected.

The input site information that also appears in the rest of the transformation is then identified, and each time the identified information appears, it is replaced with information from the relevant selected form.

In this example, the star symbol appears in the conversion effect and output site. The information from the selected form associated with the star is "3". Therefore, the star of the effect and output site is replaced with 3 as shown in FIG. It should be understood that the replacement with the number 3 of stars may not be permanent at the effect and output site, as it may only be done for the application of the transformation.

The form in the output universe that matches the output site is then selected and replaced with the effect.

In this example, the side of length 3 of the triangle in the output universe is selected and replaced with the same side, but as shown in FIG. 41, two lines are added to create another triangle.

Relative Positions Between Parts of the Transformation In one embodiment, the relative positions of the forms inside the input site, effect, and output site may affect the transformation. For example, having two objects in the same position within their respective rectangles for the output site and effect can mean that they are connected together. For example, if the effect object E is in the same location as the output site object O, O is replaced by E. If the objects are in different positions, you may get different results as shown in the example below.

For example, if the transformation shown in FIG. 40 is replaced with the transformation shown in FIG. 42, the form shown in FIG. 43 is obtained instead of the form shown in FIG. 41 .

Neuron firing In this example, the input universe and the output universe are the same, and a unique universe is shown in FIG.

The first transformation shown in FIG. 45 selects dendrites d = x and the threshold variable T and its value y as input sites. The effect is created by adding the value x to y, and as a result, the contribution of this dendrite is added to the threshold. Then, when selected by the output site, d = x and T = y are replaced with d = 0 and T = y + x, respectively.

"All" indicates that the transformation is applied to all d = x 11 times in this case until each dendrite contributes only once. Similar symbols in the transformation are considered the same object. Therefore, it is understood that d = x and T = y of the input site and the output site are the same.

The next transformation shown in FIG. 46 causes neuron firing when the threshold is greater than or equal to 6 (where 6 is the limit of this neuron). The inequality defined in the input site indicates that if this inequality is satisfied, the transformation will be applied by changing the h-labeled lightning bolt to the i-labeled lightning bolt.

The next transformation shown in FIG. 47 allows the action potential to move towards the axon terminal. In this case, the input site is empty. This means that there are "no conditions" to meet before applying the transformation. The only condition required to apply the transformation is to label the lightning bolt with an i-label and the thorny circle with an h-label in the output universe.

Here, since the action potential is close to the axon, the value a = 0 of each axon is changed to the value a = 1 as shown by the conversion in FIG. 48.

Next, after each value associated with the axon becomes 1, the symbol representing the action potential is labeled with h, as shown in the transformation of FIG.

A network of neurons can be created by connecting the axons of this neuron to the dendrites of other neuronal networks. A rough method of transmitting the value of axon a = x to the value of its associated dendrite d = y can be performed by the transformation shown in FIG.

Synaptic Cleft FIG. 51 shows a model of the synaptic cleft.

The transformation shown in FIG. 52 makes it possible to model the release of 10 neurotransmitters in the synaptic cleft.

The input range is a = x of this neuron. When a = 1, a random position within the output range is selected and a circular neurotransmitter is inserted at this position. Blank output sites do not specify a specific position and are therefore interpreted as random positions within the output range. The output range defines the area separated by the rectangular box shown in FIG. 53.

After the neurotransmitter is released, a = 1 is reduced to 0 using the transformation of FIG.
The transformation of FIG. 55 allows neurotransmitters to be moved within the synaptic clefts separated by the cylinder shown in FIG. The transformation requires a neurotransmitter and the boundary sphere of the space around it. The transformation then returns the neurotransmitter to a random location within the border sphere, unless its location within the border sphere is outside the synaptic cleft separated by a cylinder.

Finally, when the neurotransmitter binds to the neuroreceptor, its contribution is added to the dendrite value d = y, as represented by the transformation in FIG. This transformation is always present and functions when conditions are met, i.e., when a neurotransmitter binds to one of the neuroreceptors.

Construction of Graphene Sheet The transformation of FIG. 58 represents one of the rules required to assemble a graphene sheet. This conversion obtains four CO ₂ molecules and adds them to the graphene sheet shown in FIG. 59 .

CO ₂ Collector The transformation shown in FIG. 60 functions as a collector by using the functions "take" and "add". In this example, CO ₂ molecules are collected and added to the CO ₂ collection. Literally, this takes CO ₂ molecules from the input universe and moves them into a group called a carbon dioxide collection. The input universe is considered to be a container containing a large number of CO ₂ molecules.

After collecting the 5 molecules, the scenario shown in FIG. 61 is obtained.
A more complex system for collecting CO ₂ molecules to build a portion of the graphene sheet is shown in FIG. The numbers in the red box indicate the order in which the conversions are applied. First CO ₂ is collected and then the graphene sheet is expanded.

Satellite rotation transformations can be used to move and rotate objects such as 3D meshes.

Manual rotation It is possible to rotate the 3D satellite by the transformation shown in FIG. 63.

Applying this transformation to a 3D model of a satellite multiple times yields a series of states in the output universe shown in FIG.

Automatic signal finder transformations If you need a satellite that automatically directs the satellite itself to a particular signal, you can use the set of transformations shown in Figure 65, which is the angle between the satellite's central z-axis. Rotates the satellite in each plane when it differs by more than 20 degrees from the signal source in each plane.

In addition to this set of conversions, rotation in other directions when the angle is 180 or greater gives a set of conversions that allows the satellite to autonomously point its antenna at its signal. Higher accuracy can be achieved by enabling rotation and using a 1 degree threshold instead of a 20 degree threshold to make the satellite rotation increment 1 degree.

Rotations Activated by the Keyboard The application of rotations can be triggered by pressing the arrow keys on the keyboard, as shown in FIG.

User interface for rotating satellites or real physical buttons The lines shown at the transformation input site of FIG. 67 are connected to the lines in the YZ plane. Note that the satellites in the effect are considered to be in absolute position and will be replaced in this direction regardless of where the satellites were previously oriented.

Vehicles and Timelines At each step or as the system changes, the state of the system can be recorded and stored in memory. This builds a timeline for the system as described above.

When the transformation is applied, the number of steps in the system increases. In this example, it is assumed that the number of steps is increased by 1 each time the transformation is applied in the system. It should be understood that the number of steps does not necessarily have to be an integer, but can be a real number that increases with the increment of the real number. The example of a real step is for explaining the passage of time as a step. Moreover, in one embodiment of the invention, the transformation can be applied continuously rather than discretely.

This system describes a vehicle moving according to various types of transformations. Using the transformation shown in FIG. 68, the average speed of the vehicle in the x direction can be calculated between step 0 and step 50. At the input site, information about the state of the vehicle in step 0 and step 50 is accessed. A transparent car is used to indicate that the car is from the preceding step.

By accessing the timeline with transformations, it is possible to seek modeling of velocities, accelerations, derivatives and many other concepts.

Modeling and projecting the actual speed measurements of the vehicle onto the mileage meter is done using the transformations shown in FIGS. 69 and 70. In this case, "actual" means the value of the step at the moment when this transformation is about to be applied. Note that the average speed of the last 5 steps is calculated to get the current speed. The transformation in FIG. 69 allows the PresentSpeed to be inserted into the output universe.

The transformation in FIG. 70 takes the PresentSpeed from the previous output universe and writes the number of velocities in a black rectangle in blue. Next, as shown in FIG. 70 , a blue number and a square are superimposed on the image of the mileage meter. The word "addition" means that the rangefinder is not replaced and a black square with numbers is added to the image of the rangefinder.

Conversion notation Please understand that the above conversion notation is merely an example.

For example, a conversion can be described as a two-part consisting of an effect and an output site. In this case, you need to identify the form that will serve as input. This is because the conversion effects, output sites, and the form need to be used to initialize the data.

It should also be understood that transformations and structures can be displayed in different ways depending on the needs and modeling context. For example, the transformation of FIG. 72 can be replaced with the transformation shown in FIG. 73, which shows a more compact description.

When the orientation and matching transformation matches the input site with the form of the input universe, in some embodiments of the invention the match can be literal or can be rotated and transformed.

In the simple system above, the match in the output universe was literal.
In the above carbon dioxide collector, the match in the input universe is not literal. You can select molecules that are oriented in different directions.

Interconnect Input sites can connect to a number of different effects. Effects can be connected to multiple output sites. The input site can connect to multiple input universes. The output site can connect to multiple output universes.

The output universe can be a conversion, and the input universe can be a conversion.
In addition, a single input site can be connected to a complete transformation consisting of that input site, effects, output sites, and data. The input site can also connect to a set of conversions or even a universe. Similarly, a single output site can be connected to a tripartic transformation, a set of transformations, or a universe.

Graphical User Interface (GUI)
Below is a graphical user interface (GUI) that can be used to define the transformation. As shown in FIG. 74, the GUI comprises a first part where the transformation can be visually created and a second part where the output universe can be represented.

The first part of the GUI comprises five windows. In the first window, the input site can be represented visually or graphically. The second window is used to visually define the effect, while the third window is used to visually represent the form contained in the output site. The fourth window is used to visually represent the output range, and the fifth window is used to input the conversion data.

In one embodiment, the unranged transformation is displayed on the left and the universe is displayed on the right , as shown in FIG. 88.

In one embodiment, the GUI can include a third part for visually representing the input universe.

In one embodiment, the transformation and the sequence of transformations can be represented directly within the universe.

In one embodiment, the sequence of transformations can be grouped under a single representation.

In one embodiment, the sequences of transformations can be represented in 2D or using lines, arrows (or other shapes) between transformation representations to show how they are applied. ..

In one embodiment, the sequences of transformations may be placed in 3D space using lines, 3D arrows (or other shapes) between the transformation representations to show how they are applied. can.

In one embodiment, the fourth window for representing the output range may be omitted. In the same or another embodiment, the first part of the GUI can include a window for visually representing the input range.

To create the transformation, the user first imports a form such as a 3D model in the output universe from the library, or uses the "Geometry Tab" element at the top left of the GUI to create a basic 3D. Create a model. In this example, as shown in FIG. 75, the user creates a tetrahedron in the universe.

The user then selects a face from the tetrahedron and drags and drops it onto the conversion output site, as shown in FIG.

As shown in FIG. 77, the user copies the entire contents of the output site into the effect.
The user can then change the geometry of the form contained in the effect window by adding a form, using the elements of the Geometry tab, or manually changing the 3D model. Create an effect in the window. Alternatively, the user may import the 3D model from the library. In this example, as shown in FIG. 78, a smaller tetrahedron is added on the triangle.

As shown in FIG. 79, the user then inputs the conversion data. For example, the user can click the "All" radio button in the conversion data and click the "Checkmark" four times to inject this conversion into the universe. The All button indicates that the transformation is applied to all triangles in the universe that match the triangle at the output site.

When the user presses the "Step" button, the conversion is applied once. Each triangle of the tetrahedron is replaced by a triangle with a smaller tetrahedron. The resulting form is shown in FIG.

When the user presses "Play", the other three transformations are applied in succession, given the following complex fractals, resulting in the form shown in FIG. 81.

Using Transform Input Sites, Output Sites, and Effects Here, the same example is illustrated while using a transform that has input sites. In this case, text, values, 3D models, or data can be added to the universe. In this example, the text "D = 2" is added to the universe, as shown in FIG.

As shown in FIG. 83, the user adds the text "D = x" to the input site. An "x" indicates that the output site matches an object of the form "D = x". The "x" can be anything. The user then indicates where in the effect (or optionally the output site) the value "x" is used. In this example, the value "x" is assigned to the height of the smaller tetrahedron of the triangle.

After selecting the radio button "All" to inject and apply the transformation, the transformation matches the input site with the text "D = 2" from the universe. Next, as shown in FIG. 84, the value "2" is replaced with "x" in the effect. The effect of the transformation applied and the output site are calculated.

The universe as a result of applying the transformation four times is shown in FIGS. 85 to 88.
Dynamic input In the previous example, the text "D = 2" is always the same in the universe. More dynamic and variable inputs can be used.

I) The conversion can change the value "D = 2" to some larger value after each application of the conversion. For example, as shown in FIG. 89, the transformation can add a tetrahedron to the triangle while changing "D = 2" to "D = 2.5" calculated by "D = x + 0.5". can.

The first application of the transformation gives the added tetrahedron a height of 2, a second height of 2.5, a third height of 3, and so on. In other words, the height of the conversion depends on the initial value of "D" in the universe and the number of times this conversion has been applied.

II) Inputs can be extracted from other shapes in the universe that change themselves as the universe changes under the application of multiple other transformations. For example, "x" can be obtained from the distance between two points between objects.

Scope of conversion a) After performing the above steps, the user can use the output range of the conversion to output the conversion by clicking the appropriate button in the menu at the bottom right of the conversion, as shown in Figure 90. You can decide to copy all the data in the universe within range.

b) The user can select some faces of the tetrahedron of the range. By pressing the delete button, the selected face is deleted. In the example shown in FIG. 91, only two faces are held. That is, the transformation applies only to the tetrahedral faces of the universe that correspond to the faces that are still in range.

In this example, as shown in FIG. 92, after one application of the transformation, the resulting universe includes tetrahedra with smaller tetrahedra arranged on the left and bottom surfaces.

Recording a conversion Manual changes made within the universe can be recorded as a conversion.

For example, as shown in FIG. 93, the user can add a line segment with a point at one end to the universe. This can be done by using the "Geometry Tab" component.

The user presses the "Record" button at the bottom of the user interface. The conversion is created and the manual changes made by the user in the universe are recorded until the record button is no longer pressed.

The user clicks on the line segment and manually rotates it about 20 degrees. The transformation is then created and displayed on the left, as shown in FIG. 94.

The user deselects the record button if he is satisfied with the recorded conversion. Next, after entering "12" in the "Number of times" box of the conversion data, the recorded conversion is injected into the universe, as shown in FIG. 95.

After pressing "Play", the universe line segment rotates 12 times, as shown in FIG. 96.
Editing Injected Transformations Transformations injected into the universe (which can be, but are not limited to, 2D or 3D space) can be edited and modified. FIG. 97 shows an interface that presents a sequence of recorded transformations. In the conversion sequence, the user can double-click on one of the conversions.

As a result of double-clicking, the conversion is displayed in the GUI. The user can then edit a particular transformation for modification. After changing the conversion, the user can save the conversion, and when it is time for the conversion to be applied, the conversion behaves differently.

In sequences, you can drag the transformations up and down to reorder them.
In one embodiment, the object of the method and system is to provide a general purpose system that can create models, simulations, communications, and optimizations and implement them for knowledge in all areas. Based on this method, all knowledge and applications can be integrated under a single system and paradigm. Here, the general-purpose system will be referred to as a "framework" below.

The embodiments of the present invention described above are intended to be merely exemplary. Therefore, the scope of the invention is intended to be limited only by the appended claims.

Claims (21)

It is a processing unit device
At least one control unit that controls the operation of the processing unit device, and
A transformation logic unit that includes at least one transformation block associated with a transformation performed by the at least one control unit, wherein the transformation includes effects applied to the output site, including the output site form contained in the output universe. , The output universe comprises a hierarchical data structure, the effect is associated with a change in properties in a given hierarchical data substructure, and each of the at least one transformation block comprises an effect block and an output site block. The effect block includes at least one first storage unit for storing information about the effect, and the output site block contains at least one second storage unit for storing information about the output site. Including, conversion logic unit and
The user graphical interface comprises a display unit including a user graphical interface.
A first section for representing the output universe and the effect applied to the output site, including the output site form contained within the output universe.
A processing unit device comprising a second section for displaying a graphic representation of the transformation. The processing unit according to claim 1, wherein the at least one control unit is configured to further perform the conversion using a form conversion instruction set containing conversion data defining parameters for the conversion. device. The processing unit device according to claim 1, wherein the at least one control unit is included in the conversion logic unit. The processing unit device according to claim 3, wherein the at least one control unit is included in one of the at least one conversion block. The at least one control unit includes a plurality of control units, the at least one conversion block includes a plurality of conversion blocks, and each of the plurality of control units is included in each conversion block among the plurality of conversion blocks. The processing unit device according to claim 4. The processing unit device according to claim 5, wherein the at least one control unit includes at least one first logical unit included in the effect block and at least one second logical unit included in the output site block. .. The processing unit device of claim 1, wherein the information about the output universe is stored in at least one of some of the first storage units and some of the second storage units. The processing unit device according to claim 1, wherein the at least one first storage unit and the at least one second storage unit include a register. Each of the at least one conversion block comprises both the output site block and an input site block containing at least one third storage unit for storing information about the input site contained in the input universe. The processing unit device according to 1. Each of the at least one conversion block
An input site range block comprising at least one fourth storage unit for storing information about an input range defining a subspace of the input universe.
The processing unit device of claim 9, further comprising an output site range block comprising at least one fifth storage unit for storing information about an output range defining a subspace of the output universe. A method performed by a computer for programming, wherein the method performed by the computer is performed by a processing unit operably connected to a non-temporary storage medium, and a method performed by the computer. ,
A transformation comprising (1) an output site containing an output site form containing a given hierarchical data substructure and (2) at least an effect associated with a change in properties in the given hierarchical data substructure. Receiving from non-temporary storage media and
By identifying at least one form in the output universe that matches the output site form, the output universe comprises obtaining an output match to which the transformation is applied, the output universe containing a hierarchical data structure and executed by the computer. How to be done further
The effect is applied to the output match to obtain a modified output match, wherein the modified output match has at least one matched hierarchical data substructure with altered characteristics. Including, the methods performed by said computer further
To output the modified output match and
To display the modified output match, comprising displaying a user graphical interface that includes a first section for representing the output universe and a second section for displaying the graphic representation of the transformation. Is a method performed by a computer, comprising displaying the modified output match within the first section of the user graphical interface. The method performed by a computer according to claim 11, wherein the output comprises displaying a graphic display of the output universe and the modified output match within the output universe. 11. The conversion further comprises transformation data defining parameters for the transformation, and applying the effect to the retrieved output match is performed using the transformation data, claim 11. The method performed by the computer. The transformation further comprises an input site containing an input site form, the input site form comprises another given hierarchical data substructure, and the method is at least another in an input universe that matches the input site form. Further comprising obtaining an input match with the input site by identifying the form, the input universe contains another hierarchical data structure, and applying the effect to the retrieved output match is retrieved. The method performed by the computer according to claim 11, which is performed using the information related to the input match. It further comprises displaying a user graphical interface comprising a first section for representing the output universe, a second section for defining the effect, and a third section for representing the input universe. , The method performed by the computer according to claim 14. Displaying the input site in the third section of the graphical user interface.
Displaying the graphic representation of the effect in the second section of the graphical user interface.
15. The method performed by a computer according to claim 15, further comprising displaying the output site in the first section of the graphical user interface. Further comprising receiving at least one of an input range and an output range, the input range defines a subspace of the input universe, the output range defines a subspace of the output universe, said. 14. The method performed by a computer according to claim 14, wherein the input match is selected from the subspace of the input universe and the output match is selected from the subspace of the output universe. A first section for representing the output universe, a second section for defining the effect, a third section for representing the input universe, and a fourth section for displaying the output range. , And a method performed by a computer according to claim 17, further comprising displaying a user graphical interface including a fifth section for displaying the input range. Displaying the input site in the third section of the graphical user interface.
Displaying the graphic representation of the effect in the second section of the graphical user interface.
Displaying the output site in the first section of the graphical user interface.
Displaying a graphic representation of the output range in the fourth section of the graphical user interface.
15. The method performed by a computer according to claim 15, further comprising displaying a graphic representation of the input range in said fifth section of the graphical user interface. The transformation further comprises a second effect that is modified by information from the input site contained in the input universe, the transformation block further comprises an input site block, and the input site block stores information about the input site. The processing unit device of claim 1, comprising at least one second storage unit for the purpose of A non-temporary computer-readable medium comprising computer instructions that can be executed by at least one computer processor, said computer instructions.
Receives a transformation that includes (1) an output site containing an output site form containing a given hierarchical data substructure and (2) at least the effects associated with changing properties in the given hierarchical data substructure. Computer instructions and
Includes a computer instruction to obtain an output match to which the transformation applies by identifying at least one form in the output universe that matches the output site form, the output universe comprising a hierarchical data structure, said computer instruction. Furthermore,
The modified output match comprises a computer instruction to obtain a modified output match by applying the effect to the output match, wherein the modified output match is at least one matched hierarchical data substructure with modified characteristics. The computer instructions further include
A computer instruction that outputs the modified output match, and
Includes a computer instruction to display a user graphical interface that includes a first section for representing the output universe and a second section for displaying the graphic representation of the transformation, and outputs the modified output match. That is, a non-temporary computer-readable medium comprising displaying the modified output match within said first section of the user graphical interface.