WO2015143303A1 - Systèmes et procédés destinés à la fourniture d'un produit de visualisation - Google Patents

Systèmes et procédés destinés à la fourniture d'un produit de visualisation Download PDF

Info

Publication number
WO2015143303A1
WO2015143303A1 PCT/US2015/021721 US2015021721W WO2015143303A1 WO 2015143303 A1 WO2015143303 A1 WO 2015143303A1 US 2015021721 W US2015021721 W US 2015021721W WO 2015143303 A1 WO2015143303 A1 WO 2015143303A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
animation
visual
visualization
digital
Prior art date
Application number
PCT/US2015/021721
Other languages
English (en)
Inventor
Gael-Christophe Garth MCGILL
Original Assignee
Digizyme, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US14/220,702 external-priority patent/US20150269870A1/en
Priority claimed from US14/220,730 external-priority patent/US20150269855A1/en
Priority claimed from US14/220,616 external-priority patent/US20150269765A1/en
Priority claimed from US14/220,718 external-priority patent/US20150269849A1/en
Priority claimed from US14/220,647 external-priority patent/US20150269763A1/en
Priority claimed from US14/662,888 external-priority patent/US9875567B2/en
Application filed by Digizyme, Inc. filed Critical Digizyme, Inc.
Publication of WO2015143303A1 publication Critical patent/WO2015143303A1/fr

Links

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B7/00—Electrically-operated teaching apparatus or devices working with questions and answers
    • G09B7/02—Electrically-operated teaching apparatus or devices working with questions and answers of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00—Electrically-operated educational appliances
    • G09B5/02—Electrically-operated educational appliances with visual presentation of the material to be studied, e.g. using film strip

Definitions

  • the invention generally relates to the communication, teaching, and assessment of scientific concepts and particularly to systems and methods for constructing visualization products.
  • Visual representations are starting to be used in conjunction with textual-based approaches to facilitate a person's leaning of a scientific concept. For example, illustrations, diagrams, animations, and interactive learning tools are used in classrooms and at scientific gatherings to make sense of molecular and cellular phenomena.
  • the visual tools currently being used are not necessarily well-suited to the scientific concepts being taught.
  • many currently used visual representations represent scientific phenomena with deceptive clarity, offering an oversimplified, sometimes inaccurate, explanation of a scientific concept for the sake of clarity. In that scenario, a person may recall a sequence of events but retain only a superficial understanding of the overall concept. In the case of inaccuracies, the person's foundation for future learning is compromised by an improper perception of the concept.
  • other currently used visual representations introduce extraneous complexity not relevant to a learning goal, which may be equally misleading. Regardless of the content of the visual representation, such representations are often borrowed from independent sources and not contextualized to the scientific concept or to the audience.
  • the invention recognizes that when strategically designed to an educational level of an audience, with a clear learning objective in mind, scientific visualizations are powerful tools for describing the intricacies of scientific concepts. Perhaps more than in any other area of science, the systems and methods of the invention may help people grasp the complexity of cellular and molecular events that are both too small to see with the naked eye or microscope or too rapid to observe. Accordingly, the invention provides a learning approach for representing complex events in space and time with narrative visual representations that are more engaging and memorable than other forms for communicating scientific concepts.
  • aspects of the invention are accomplished with methods and systems that can be used to visually communicate a scientific concept by building a visualization product (using pre-made or de novo constructed assets) to represent entities in a visualization.
  • the assets include models with animation rigs that confer scientifically accurate animation dynamics on the modeled objects.
  • the narrative visualization can present an animation in which the depicted objects move according to what is known to be scientifically possible.
  • the visualization product can be an interactive display that realistically models scientific phenomenon according to user input, allowing users to see and comprehend the immense variety of possible outcomes.
  • the visualization can depict entities such as molecules, cells, organisms, embryos, solar systems and galaxies, or fluid systems, for example. A user can change conditions or inputs and observe the effects on the depicted entities to see, for example, all of the possible outcomes of interest.
  • the entities are animated by models built from scientific data, and the models can be rigged according to animation rigging principles, the entities as depicted within the visualizations will represent natural phenomenon with the best scientific accuracy possible.
  • animation according to these principles allows the modeling and rigging to be uncoupled from telling the story, a teacher or scientist can imagine a story and compose the visualization product using digital assets that are separately modeled and rigged. This allows a teacher or scientist to design or create accurate and high-quality interactive animations, tailored to an education level of an audience, without being proficient in 3D modeling or animation rigging.
  • the invention provides tools by which a person can effectively and accurately communicate descriptions of natural phenomenon, without themselves having to become experts in modeling and animation.
  • the visualization products provided herein can be used in classrooms and scientific publishing to make learning a primarily visually driven approach.
  • the invention provides a method for providing a visualization product that includes constructing, using a computing device including a processor coupled to a non- transitory memory, an electronically displayable visualization product.
  • the visualization product includes at least one digital asset that visually conveys at least a portion of a scientific concept.
  • the digital asset has data representing a structure and a rig that defines animation dynamics for the structure, and is tailored based on an education level of an audience.
  • the method includes providing the visualization product for viewing by the audience on an electronic display device.
  • the audience may be any single person or group of people with any education level, for example, students (e.g., K-12, college, graduate, etc.), researchers working on scientific studies in academia or industry, the lay public (e.g., as viewers of an educational show or readers of a book), or other people.
  • the method includes receiving data related to the education level of the audience.
  • the provided visualization product may include a single digital asset and the single digital asset may visually convey an entire biological concept.
  • the visualization product includes a plurality of digital assets. Each of those digital asset may convey a portion of a biological concept, which in the aggregate, s convey the entire scientific concept.
  • the visualization product may be or include an animation such as a video or a 3D, interactive animation, one or a set of still images, a game, or another model -based visual medium.
  • the visual product can be used to convey any of a variety of scientific concepts and can address the notion of threshold concepts (topics that a learner must master without which progressing in a field of science would not be successful).
  • biological entities can be shown, such as an enzyme and its substrate or a multi-protein signaling pathway.
  • Gene flow between two or more different populations can be illustrated, or the interaction between a chemical and an organism such as showing the effect of a drug on a developing embryo.
  • Digital assets can be tailored to an educational level of an audience of the visual product. For example, tailoring can be done by automatically visually concealing one or more portions of the digital asset based on the data related to the education level of the audience.
  • Visual products may include an assessment component.
  • a visual product includes an embedded, adaptive assessment module.
  • An audience views parts of an animation and their understanding is evaluated through their interactions (e.g., drag the right substrate to the enzyme). As the audience demonstrates mastery of the topic, the animation progresses to the next level or subject.
  • the assessment can include a back-end reporting component and thus can, for example, provide an educator with a score report or provide a user with a progress report for self-evaluation.
  • the invention offers a system for providing visual scientific content.
  • the system includes a processor coupled to a non-transitory memory.
  • the system is operable to construct an electronically displayable visualization product that includes at least one digital asset that visually conveys at least a portion of a scientific concept.
  • the digital asset as used within the system includes data representing a structure and a rig that defines animation dynamics for the structure.
  • the system can be used to tailor the digital asset to an education level of an audience.
  • the system provides the visualization product for viewing by the audience on an electronic display device.
  • the invention provides a curated model database.
  • the invention provides a database of scientifically accurate digital models that are scene ready to be used in a static image, animation or interactive.
  • databases of the invention allow for a do-it-yourself approach in which a user can easily and cost effectively build an animation that depicts scientific concepts, such as concepts of cell and molecular biology, that are properly contextualized to an audience.
  • the database serves as a starting-point not only for an experienced scientist-animator (who can save time and increase the accuracy of their work), but also for a person who has a scientific background but lacks any experience in animation or visual arts (in that the system allows them to easily build a customized scientific image, animation or interactive without any technical knowledge).
  • aspects of the invention are accomplished by using a curated database of models in which each model includes scientifically accurate structural data as well as an animation rig defining dynamics for the modeled structure.
  • the structural data and the rigs are built based on scientific information.
  • the curated models can be used in creating images, animations or interactives.
  • the structural data provides that depicted objects will be scientifically accurate and the animation rigs provide scientifically accurate range-of-motion or dynamic information so that the animations will illustrate interactions accurately. Since the structures and the rigs are curated and stored in a database, users can use them as-is— that is, the curated models are "ready for use” in building animations. The user need not manipulate the files to confer accurate dynamics on the depicted structures.
  • Entries from the curated model database can be imported into an animation platform to create animations that may be used, in turn, to create digital media such as still images, videos, games, simulations, or other interactive media.
  • a student or educator can use the curated model database to create media that depict scientific phenomena that are being studied. Since the database entries use data sourced from scientific studies, accuracy is not sacrificed when building visual media.
  • the invention provides a system for building a life science image, animation or interactive that includes a processor coupled to a non-transitory memory.
  • the system stores a plurality of digital models.
  • Each digital model has data representing a structure and a rig that defines animation dynamics for the structure such that a range of motion of each digital model on an electronic display device is predetermined without manipulation from a user.
  • the data and the rig of each digital model govern how the modeled structure will interact with other modeled structures in a visualization.
  • the system may include a graphical interface that allows a user to build an animation comprising one or more of the models.
  • the system may include features to aid a user in understanding the underlying science and the modeling and rigging decisions that went into building the model.
  • the models may include citations, to give a source for the data.
  • Models may include hypothetical data representing an inference about a region of the biological structure lacking scientific data. This can aid a user in understanding an overall system even where, for example, some short piece of a protein sequence is yet to be finally determined.
  • Systems of the invention may also allow a user to create and input new digital assets themselves.
  • a rig may define animation dynamics with different sets of rules for each digital asset based on surrounding biological environmental conditions (e.g., temperature, salinity, pH, osmolality, viscosity, or others).
  • the rigging techniques are used to understand protein conformations.
  • a protein may be given a rig that allows the data to illustrate a biological entity in a plurality of realistic conformations within the animation.
  • the methods can include providing hypothetical data representing an inference about a region of a biological structure for which scientific data is not available.
  • the rig allows the data to illustrate a protein in a plurality of realistic conformations within the animation.
  • a curated model database as provided by the invention may be used to depict scientific phenomenon such as a cascade of events in which at least two depicted biological structures interact only indirectly.
  • the invention provides a method of creating and providing a repository of curated models, which can be used, for example, to convey a scientific concept.
  • a plurality of digital models are stored in a non-transitory computer-readable medium, allowing them to be used to visually convey to an audience a scientific concept, each model including data representing a structure and a rig that defines animation dynamics for the structure such that motion of each digital model on an electronic display device is predetermined without manipulation from a user.
  • the plurality of digital models may be used to convey a scientific concept to the audience or to build an animation comprising the plurality of digital models.
  • the invention provides systems that allow users to design and model natural structures such as biomolecules.
  • the system can include a collection of individual models and also provide users with the option to select certain simulation/interaction modalities that will influence the dynamics of models within a simulation created by the user.
  • the system may include environments, such as pre-curated biological or cellular environments that allow the users to simulate and test the interactions of the natural structures with their environments. For example, a team of users could design a novel therapeutic protein and model that protein within the context of a cellular membrane which itself has a realistic and accurate composition of lipids and cell-surface proteins. In such an example, the users could select a molecular dynamics modality to govern the motions and interactions of the novel therapeutic and the cell surface proteins.
  • the system presents the users with the relevant user interface and molecular pre-sets and also facilitates the automated and sophisticated creation of a simulation or visualization of the modeled natural structures.
  • the users themselves can view and interact with the simulation or the product can be provided for access by third parties.
  • a feature of the invention is the ability for groups of users to collaborate, even over diverse geography and time.
  • a first user could set up the question to be explored and another user or team could provide the individual models, such as proteins, nucleic acids, or crystals.
  • a different set of people could provide interesting modeling algorithms and the entire collaborative process could even feedback on itself, with different parts of the team making changes along the way based on extrinsic and intrinsic insights.
  • systems of the invention use scientifically accurate models of natural structures to create simulations in which users select or control the environment and the modeling algorithms, researchers can use the system to conceptualize, design, model, and simulate natural phenomena involving biomolecules or other actors. Novel combinations of elements can be made and the system will simulate how those actors interact and what results they produce. Thus, systems and methods of the invention provide tools for the synthesis of natural information.
  • the invention provides an integrated system for providing a scientific simulation.
  • the system includes a processor coupled to a non-transitory memory.
  • Stored within the system is at least one digital model that includes data representing a structure and a rig that defines animation dynamics for the structure such that a range of motion of the at least one digital model on an electronic display device is predetermined without manipulation from a user.
  • They system also includes at least one molecular dynamics modality selectable by the user that applies one or more parameters that influence the animation dynamics of the structure in a simulation provided by the system.
  • the user- selectable molecular dynamics modality may include, for example, Brownian dynamics, a Monte Carlo simulation, an explicit- solvent coarsegrained simulation, an implicit- solvent coarse-grained simulation, or molecular dynamics.
  • the structure and the rig of the digital model and the selected molecular dynamics modality may be used to govern how that digital model will interact with other digital models in the simulation and a resulting visualization.
  • the rig that defines animation dynamics may include different sets of rules for the digital model based on the selected molecular dynamics modality.
  • the rig may offer various methods and levels of coarse-graining detail and the selected molecular dynamics modality can allow the user to control one or more of temperature, salinity, pH, osmolality, or viscosity in the simulation.
  • the rig allows the data to illustrate a biological entity in a plurality of realistic conformations within the simulation.
  • the system may offer relevant simulation methods and do so in a way that alerts the user to the
  • the system can help users in the design of novel molecular entities by offering a function-based menu of molecular domains. For example, if a user is designing a cytoplasmic protein and needs to properly localize that protein to a membrane for the protein to function properly, the user could select a 'membrane recruitment' or
  • the GUI could include a molecular domains/parts menu and a number of relevant structures/rigs could be provided there, ready for modeling and appending to the rest of the molecule being designed (for example, like PH or C protein domains, or a farnesyl chemical group).
  • the system preferably includes pre-modeled or pre-populated (but still customizable) environments within or against which a model can be simulated. This may be used to provide a 'panel' of controls against which to simulate novel molecular entities. For example, if a new protein therapeutic has been modeled, it can be examined for binding to a receptor on a surface chosen by user (such as a colon cancer cell plasma membrane). However, the new protein therapeutic could further be simulated against a panel of other common cellular (and inorganic surfaces such as those found on medical devices) to model and predict any adverse interactions. In other words, a researcher could probe such questions as "is this protein sticking to the surface of normal colon epithelium as well?" or "is this protein interacting with artificial surfaces like PLGA?”
  • Systems and methods of the invention may be used to integrate multiple steps of what is currently experienced as a discontinuous process of modeling, simulation, and visualization.
  • the invention facilitates use by a new type of user (e.g., users who wouldn't otherwise know how to do this type of work) thereby democratizing the use of modeling and simulation in molecular/cell biology. Since the system includes built-in menus for parameterizing simulations, the invention overcomes prior-art problems in which knowing which simulations to run and how to
  • systems and methods of the invention provide the power and advantage of integration and presets at all stages— modeling, simulation and visualization.
  • aspects of the invention provide a method for providing a scientific simulation.
  • the method includes obtaining a plurality of digital models stored in a non-transitory computer- readable medium.
  • Each digital model has data representing a structure and a rig that defines animation dynamics for the structure such that motion of each digital model on an electronic display device is predetermined without manipulation from a user.
  • At least one molecular dynamics modality is selected that applies one or more parameters that influence the animation dynamics of the plurality of digital models and a simulation is generated through use of the plurality of digital models and the selected molecular dynamics modality.
  • the simulation is output for display on an electronic device as a visualization that conveys a scientific concept to a user.
  • the user- selectable modality may provide, for example, Brownian dynamics, a Monte Carlo simulation, an explicit- solvent coarse-grained simulation, an implicit-solvent coarsegrained simulation, an all-atom simulation, or a quantum mechanical simulation.
  • the selected molecular dynamics modality and the rig of the digital model govern how that digital model will interact with other digital models in a visualization.
  • At least one of the digital models comprises two or more alternative rigs.
  • the memory includes an environment model that the user may include in the simulation (e.g., an environment model that represents at least a portion of a biological lipid membrane).
  • the method may include receiving a new digital model from the user and creating the simulation using the environment model, the new digital model, and the selected molecular dynamics modality.
  • the simulation may, for example, include a number of the digital models in a pathway animation depicting a cascade of events in which at least two depicted biological structures interact only indirectly.
  • the selected molecular dynamics modality allows the user to control one or more of— for example— temperature, salinity, pH, osmolality, or viscosity.
  • a user may create the simulation and share a visual version of the simulation with a plurality of viewers.
  • the system includes a plurality of digital models stored in a non-transitory computer-readable medium , each digital model comprising data representing a structure and a rig that defines animation dynamics for the structure such that motion of each digital model on an electronic display device is predetermined without manipulation from a user.
  • the system includes one or more user-selectable molecular dynamics modality that applies one or more parameters that influence the animation dynamics of the plurality of digital models.
  • the system is operable to generate a simulation through use of the plurality of digital models and the selected molecular dynamics modality and output the simulation for display on an electronic device as a
  • the system may further include user- selectable environment models that the user may include in the simulation.
  • the selected molecular dynamics modality and the rig of the digital model may govern how that digital model will interact with other digital models in a visualization.
  • the user may create the simulation and share a visual version of the simulation with a plurality of viewers.
  • the system may be operated to allow a plurality of users to contribute to the simulation by selecting one or more molecular dynamics modalities or digital models for inclusion in the simulation.
  • Systems and methods of the invention provide systems and methods or use of digital assets.
  • the invention provides a visualization product that includes digital assets that visually convey a scientific concept according to an educational objective.
  • Each digital asset is scientifically accurate and a number of the digital assets work together to, in the aggregate, provide a visual narrative that visually conveys the scientific concept without including simplifications or inaccuracies that hinder understanding or seed misconceptions.
  • the digital assets can be built with rigged models that include scientific data to represent structures and rigging based on computer animation principles to define motion dynamics for those structures.
  • computer animation principles may be employed to give life to structural data and to produce the digital assets.
  • the visualization product can included educational materials such as, for example, an integrated assessment tool and can be provided as animations or other formats. Since the visualization product teaches scientific concepts with accuracy and without misconceptions, people can assimilate the dynamic and increasingly complex interrelated processes to understand the underlying phenomena.
  • the invention provides a visualization product that includes a plurality of digital assets stored in a non-transitory computer-readable medium that in the aggregate provides a visual narrative that visually conveys to an audience at least a portion of a scientific concept.
  • Each digital asset is scientifically accurate and tailored based on data related to an education level of the audience (e.g., K-12, college, post-doc).
  • the plurality of digital assets may be viewed as an animation at a first level of detail related to a first age level and a second level of detail related to a second age level. Tailoring the digital asset may include visually concealing one or more portions of the digital asset based on the data related to the education level of the audience.
  • At least one of the plurality of digital assets comprises a model that includes data representing a structure and an animation rig to control dynamics of the animation.
  • the digital asset is generated based on scientifically accurate data representing a biological structure, wherein portions of the biological structure for which scientifically accurate data does not exist are represented within the digital asset.
  • the plurality of digital assets may be rendered in an animation.
  • the visualization product may include educational materials such as an assessment tool.
  • the plurality of digital assets may be tailored according to an educational standard.
  • the digital assets may include one or more static images, one or more animations, one or more interactive images, a progression of images, an interactive animation, a game, a three- dimensional model, a simulation, and a combination thereof.
  • the non- transitory computer-readable medium is part of a server computer system making the plurality of digital assets available for download to a personal computer.
  • the scientific concept is a signaling pathway, and the plurality of assets in the aggregate convey a progression through the pathway.
  • aspects of the invention provide a method of conveying a scientific concept by obtaining a plurality of digital assets stored in a non-transitory computer-readable medium that in the aggregate visually conveys to an audience a scientific concept, each digital asset being scientifically accurate and tailored based on data related to an education level of the audience (e.g., by visually concealing one or more portions of the digital asset based on the data related to the education level of the audience).
  • the plurality of digital assets is used to convey a scientific concept to the audience.
  • One or more of the digital assets may be an animation that includes a model comprising data describing a structure and a rig to control dynamics of the structure.
  • the method may include selecting a level of detail at which at least one of the digital assets is to be displayed.
  • Embodiments of the invention include evaluating a student' s understanding of the scientific concept through an assessment tool, which may be embedded within the plurality of digital assets.
  • invention provides a visual cell.
  • the invention provides a platform that visually integrates substantially all components and processes of a single cell.
  • Systems and methods of the invention guide a person through the visualizations of the components or processes in the cell in a manner that illustrates to the person how the components and processes are connected to each other. In that manner, systems and methods of the invention contextualize the components and processes within the cell, allowing a person to view the cell as a series of interrelated components and processes, rather than a set of discrete and unconnected ideas.
  • aspects of the invention are accomplished using interconnected narrative visualization digital assets to depict substantially all components and processes of modeled entities.
  • the visualizations are tailored to an education level of an audience.
  • Individual components are modeled from scientifically accurate data and animation and simulation techniques are used to illustrate the actions and interactions of those components that make up the biological processes related to the cell. Levels of detail and presentation styles may be automatically tailored to the education level.
  • Systems and methods of the invention are not limited to the cell, and can portray an entire biological entity, such as an organ, a biological system, an organism, as well as research methods and model organisms used to study cells. Additionally, systems and methods of the invention may be used to portray a solar system, lab equipment, a machine, or other physical or natural phenomenon.
  • the learning digital assets are narrative visualization in that they do not require expository prose to communicate the scientific concepts.
  • systems of the invention provide visualizations that can be viewed, browsed, and interacted with. Since all of the systems and sub-systems of the biological entity are included and shown with scientific accuracy, the audience can satisfy their interest and curiosity and gain an understanding of the biological entity. The audience can view the interplay of processes and components such as metabolism, reproduction, injury, infection, mutation, or recombination. The audience can thus learn how living things grow and propagate themselves and how life adapts and changes over time. Since the biological entities are modeled visually and the models are tailored to the audience's education level, biological concepts are taught holistically and effectively. Since original scientific data is used in building and animating the models, misleading simplifications are avoided.
  • the modeled biological entity is a cell
  • the invention provides systems and methods for viewing components and processes of the cell.
  • the cell can be represented visually, with substantially all of its components and processes included.
  • the visualizations depict, for example, the lipid membrane, the cytoplasm, and all the mechanics of replication, transcription, and translation.
  • At least one of the structures is represented by a rigged model that is built with structural data and an animation rig that confers a scientifically-accurate range of motion on that structure.
  • Components are also simulated using various methods (including Molecular Dynamics, Brownian Dynamics and other coarser-grained methods).
  • the whole-cell can be used in a visualization product such as a computer-based interactive cell within which a user can zoom and pan to watch reactions.
  • Systems and methods of the invention may include animations, images, interactives, games, or other such educational media.
  • the invention provides a system for visualizing substantially all components and processes of a single cell, such as a prokaryotic or eukaryotic cell.
  • the system includes a processor coupled to a non-transitory memory and stores a plurality of interconnected visualizations that in the aggregate represent substantially all components and processes of a single biological entity, such as a single cell.
  • Each visualization uses at least one digital asset that visually conveys at least a portion of a component or process associated with the biological entity.
  • At least one of the digital assets includes data representing a biological structure and rig that defines animation dynamics for the biological structure.
  • a single digital asset may convey one entire biological concept.
  • the system can present the visualizations to a user with a consistent visual style.
  • the system can tailor at least one of the visualizations based on data received to the system related to an education level of a user (e.g., from within K-12, college, graduate, or post-doctoral). For example, the system may present at least one of the visualizations at a first level of complexity corresponding to a first educational level and at a second level of complexity corresponding to a second educational level. Tailoring may be done by concealing one or more portions based on the data related to the education level of the audience.
  • a user may self-select a level of complexity at which to view the visualizations prior to entering the system, and may also self- select a level of complexity at each visualization. That is, a user may choose to change the level of complexity for any single visualization to simplify or increase the complexity of the visualization based on their understanding of the cellular concept they have just viewed.
  • the system can portray a variety of biological systems and subsystems relating to, for example, metabolism, genetics, signaling, injury and repair, and other phenomena.
  • the visualizations may depict an interaction of at least two biological entities.
  • At least one of the visualizations may depict a signaling pathway of biomolecules in which at least two interact only indirectly.
  • the rig allows the data to illustrate a protein in a plurality of realistic conformations. The entities and the interaction will be governed by the animation rig or a simulation method.
  • the system may leave regions the biological structure in at least one of the visualizations undefined for which scientific data is not available.
  • the rig that defines animation dynamics will use different sets of rules for each digital asset based on surrounding biological environmental conditions (e.g., pH, T, salinity, osmolality, etc.) for a curated model within an visualization digital asset.
  • surrounding biological environmental conditions e.g., pH, T, salinity, osmolality, etc.
  • the system may include or use any suitable hardware.
  • a server computer or a cloud system i.e., multiple storage devices distributed across a cloud computing system
  • the visualization may be provided for viewing on an electronic device such as, for example, through a dedicated application on a tablet computer (e.g., as an "e-book” or "app") or for viewing via a web browser on a user computer device.
  • the invention provides systems and methods for interacting with a visual cell.
  • the invention provides a platform that visually integrates substantially all components and processes of a single cell.
  • Systems and methods of the invention guide a person through the visualizations of the components or processes in the cell in a manner that illustrates to the person how the components and processes are connected to each other. In that manner, systems and methods of the invention contextualize the components and processes within the cell, allowing a person to view the cell as a series of interrelated components and processes, rather than a set of discrete and unconnected ideas.
  • aspects of the invention are accomplished using interconnected narrative visualization modules to depict substantially all components and processes of modeled entities.
  • the visualizations are tailored to an education level of an audience.
  • Individual components are modeled from scientifically accurate data and animation and simulation techniques are used to illustrate the actions and interactions of those components that make up the biological processes related to the cell. Levels of detail and presentation styles may be automatically tailored to the education level.
  • Systems and methods of the invention are not limited to the cell, and can portray an entire biological entity, such as an organ, a biological system, an organism, as well as research methods and model organisms used to study cells. Additionally, systems and methods of the invention may be used to portray a solar system, lab equipment, a machine, or other physical or natural phenomenon.
  • the visualizations are narrative visualization in that they do not require expository prose to communicate the scientific concepts.
  • systems of the invention provide visualizations that can be viewed, browsed, and interacted with. Since all of the systems and sub-systems of the biological entity are included and shown with scientific accuracy, the audience can satisfy their interest and curiosity and gain an understanding of the biological entity. The audience can view the interplay of processes and components such as metabolism, reproduction, injury, infection, mutation, or recombination. The audience can thus learn how living things grow and propagate themselves and how life adapts and changes over time. Since the biological entities are modeled visually and the models are tailored to the audience's education level, biological concepts are taught holistically and effectively. Since original scientific data is used in building and animating the models, misleading simplifications are avoided.
  • the modeled biological entity is a cell
  • the invention provides systems and methods for viewing components and processes of the cell.
  • the cell can be represented visually, with substantially all of its components and processes included.
  • the visualizations depicts, for example, the lipid membrane, the cytoplasm, and all the mechanics of replication, transcription, and translation.
  • At least one of the structures is represented by a rigged model that is built with structural data and an animation rig that confers a scientifically-accurate range of motion on that structure.
  • Components are also simulated using various methods
  • the whole-cell can be used in a visualization product such as a computer-based interactive cell within which a user can zoom and pan to watch reactions.
  • Systems and methods of the invention may include animations, images, interactives, games, or other such educational media.
  • the invention provides a system for visualizing substantially all components and processes of a single cell, such as a prokaryotic or eukaryotic cell.
  • the system includes a processor coupled to a non-transitory memory and stores a plurality of interconnected visualizations that in the aggregate represent substantially all components and processes of a single biological entity, such as a single cell.
  • Each visualization uses at least one digital asset that visually conveys at least a portion of a component or process associated with the biological entity.
  • At least one of the digital assets includes data representing a biological structure and rig that defines animation dynamics for the biological structure.
  • a single digital asset may convey one entire biological concept.
  • the system can present the visualizations to a user with a consistent visual style.
  • the system can tailor at least one of the visualizations based on data received to the system related to an education level of a user (e.g., from within K-12, college, graduate, or post-doctoral). For example, the system may present at least one of the visualizations at a first level of complexity corresponding to a first educational level and at a second level of complexity corresponding to a second educational level. Tailoring may be done by concealing one or more portions based on the data related to the education level of the audience.
  • a user may self-select a level of complexity at which to view the visualizations prior to entering the system, and may also self- select a level of complexity at each visualization. That is, a user may choose to change the level of complexity for any single visualization to simplify or increase the complexity of the visualization based on their understanding of the cellular concept they have just viewed.
  • the system can portray a variety of biological systems and subsystems relating to, for example, metabolism, genetics, signaling, injury and repair, and other phenomena.
  • the visualizations may depict an interaction of at least two biological entities.
  • At least one of the visualizations may depict a signaling pathway of biomolecules in which at least two interact only indirectly.
  • the rig allows the data to illustrate a protein in a plurality of realistic conformations. The entities and the interaction will be governed by the animation rig or a simulation method.
  • the system may leave regions the biological structure in at least one of the visualizations undefined for which scientific data is not available.
  • the system may include or use any suitable hardware.
  • a server computer or a cloud system i.e., multiple storage devices distributed across a cloud computing system
  • the visualization may be provided for viewing on an electronic device such as, for example, through a dedicated application on a tablet computer (e.g., as an "e-book” or "app") or for viewing via a web browser on a user computer device.
  • the method includes accessing via an electronic display device a system for visualizing substantially all components and processes of a single cell and navigating on the electronic display device through at least one of the visualizations in order to view a component or process of the cell.
  • the system includes a processor coupled to a non-transitory memory having stored therein a plurality of interconnected visualizations that in the aggregate represent substantially all components and processes of a single cell.
  • Each visualization uses at least one digital asset that visually conveys at least a portion of a component or process within the cell, and at least one of the digital assets in at least one of the visualizations includes data representing a biological structure and rig that defines animation dynamics for the biological structure.
  • the system also includes instructions executable by the processor to cause the system to tailor at least one of the visualizations based on data received to the system related to an education level of a user.
  • Educational tailoring can include navigating through the at least one of the visualizations at a first level of complexity corresponding to a first educational level and at a second level of complexity corresponding to a second educational level.
  • Interacting with the visual cell may include searching for and retrieving the at least one of the visualizations based on subject matter.
  • Interacting may include using the system for performing a search for a biological concept, thus causing the system to display a subset of the plurality of interconnected visualizations that relate to the biological concept.
  • Interacting with the visual cell may include choosing at least one option, thereby causing the system to present a further choice, further wherein a series of choices causes the system to display a subset of the plurality of interconnected visualizations.
  • Interacting with the visual cell may include following a curriculum (e.g., designed to support an educational standard) in which the plurality of interconnected visualizations are organized to support an educational objective.
  • the plurality of interconnected visualizations may be predetermined based on an educational standard.
  • the system may select a visualization based on the educational standard.
  • the plurality of interconnected visualizations may, for example, define a lesson plan.
  • Interacting may include viewing information about the source of the supporting data for each visualization.
  • the plurality of interconnected visualizations have a consistent visual style.
  • the visualizations depict interactions of biological entities.
  • the visualizations may depict a protein in a plurality of realistic conformations, a signaling pathway comprising a plurality of biomolecules at least two of which interact only indirectly, or molecular crowding. Viewing the visualizations may be done via a user computer device such as a tablet with a viewing application or through a web browser.
  • FIG. 1 shows an overall architecture of systems of the invention.
  • FIG. 2 represents a rigged model
  • FIG. 3 diagrams a method for providing a curated model database.
  • FIG. 4 illustrates components of a computer system of the invention.
  • FIG. 5 shows a modeling and animation tool as presented by the system.
  • FIG. 6 shows a protein model according to embodiments of the invention.
  • FIG. 7 depicts a model representing a reovirus sigmal protein.
  • FIG. 9 shows a rigged model
  • FIG. 10 shows a motion of a model based on the rigging.
  • FIG. 11 illustrates protein dynamics at four different levels.
  • FIG. 12 diagrams a method of building asset database.
  • FIG. 13 diagram a method for constructing a visual product.
  • FIG. 14 depicts an interface for using systems of the invention.
  • FIG. 15 shows a method for creating a visual product.
  • FIG. 16 depicts a storyboard.
  • FIG. 17 illustrates a visual product that has been created using storyboard.
  • FIG. 18 shows a DNA strand.
  • FIG. 19 illustrates use of a modeling product to prepare a model.
  • FIG. 20 shows the layers of a layered structure.
  • FIG. 21 presents a digital asset created using layered structure.
  • FIG. 22 illustrates a digital asset that includes a photorealistic image.
  • FIG. 23 shows modeling different regions across a membrane.
  • FIG. 25 shows use of an electronic device to view a visualization product.
  • FIG. 26 illustrates use of an electronic device to interact with a model.
  • FIG. 28 shows a modeled cell with substantially all macromolecules present.
  • FIG. 30 illustrates a detail of a membrane.
  • FIG. 33 illustrates chemical structures
  • FIG. 34 shows phospholipids in leaflets.
  • FIG. 35 illustrates a structural difference
  • FIG. 36 illustrates an Archaeal monolayer.
  • FIG. 37 illustrates types of transport.
  • FIG. 38 shows an overview of membrane-enclosed organelles.
  • FIG. 39 shows endo- and exo- cytosis.
  • FIG. 40 shows a hemi-fusion intermediate
  • FIG. 41 diagrams a method for providing a scientific simulation.
  • FIG. 42 is a diagram for planning a simulation.
  • FIG. 44 shows a workflow for parameterization of novel coarse-grained models.
  • FIG. 45 gives a workflow for parameterization of novel molecules from an ab initio quantum simulation following essentially similar logic.
  • FIG. 48 illustrates a table breaking out components of the CAR molecule.
  • FIG. 49 shows a graphical interface by which a user may select a model, a molecular dynamics modality, or an environment and build the simulation.
  • FIG. 50 diagrams a biomolecular design ecosystem (BIODE).
  • FIG. 51 illustrates a simulation being provided as a visualization on an electronic device.
  • Systems and methods of the invention allow users to collaborate, design, simulate, and model natural structures such as biomolecules using a collection of individual models and providing users with the option to select certain simulation/interaction modalities that will influence the dynamics of models within a simulation created by the user.
  • systems and methods of the invention may provide for the inclusion of environments, such as pre-curated biological or cellular environments that allow the users to simulate and test the interactions of the natural structures with their environments.
  • environments such as pre-curated biological or cellular environments that allow the users to simulate and test the interactions of the natural structures with their environments.
  • a team of users could design a novel therapeutic protein and model that protein within the context of a cellular membrane which itself has a realistic and accurate composition of lipids and cell-surface proteins.
  • the users could select a molecular dynamics modality to govern the motions and interactions of the novel therapeutic and the cell surface proteins.
  • FIG. 41 diagrams a method 4101 for providing a scientific simulation.
  • the method 4101 includes obtaining 4121 a plurality of digital models stored in a non-transitory computer- readable medium.
  • Each digital model has data representing a structure and a rig that defines animation dynamics for the structure such that motion of each digital model on an electronic display device is predetermined without manipulation from a user. See FIGS. 1-2 and 8-10 and accompanying discussion below.
  • the structure and the rig of the digital model and the selected molecular dynamics modality govern how that digital model will interact with other digital models in the simulation and a resulting visualization.
  • the method can include using the rig to illustrate a biological entity in a plurality of realistic conformations within the simulation.
  • At least one molecular dynamics modality is selected 4127 that applies one or more parameters that influence the animation dynamics of the plurality of digital models and a simulation is generated 4139 through use of the plurality of digital models and the selected molecular dynamics modality.
  • the user- selectable molecular dynamics modality may provide, for example, Brownian dynamics, a Monte Carlo simulation, an explicit- solvent coarse-grained simulation, an implicit-solvent coarse-grained simulation, an all atom simulation, a quantum mechanical simulation, or a combination thereof.
  • the selected molecular dynamics modality and the rig of the digital model govern how that digital model will interact with other digital models in a visualization.
  • the user or collaborating users select 4131 at least one user- selectable environment, e.g., to control compositional and quantitative settings.
  • the simulation may be offered as a collaborative process and method 4101 may include allowing a plurality of users to contribute 4135 to the simulation by selecting one or more molecular dynamics modalities or digital models for inclusion in the simulation.
  • the simulation is output 4143 for display on an electronic device 125 as a visualization 129 that conveys a scientific concept to a user.
  • the simulation or visualization that is output 4143 is scientifically accurate and provides a powerful learning and discovery tool since it uses underlying scientific structural data and modeling or interaction modalities.
  • systems of the invention provide the users with the relevant user interface and molecular pre-sets and also facilitate the automated and sophisticated creation of a simulation or visualization of the modeled natural structures.
  • the users themselves can view and interact with the simulation or the product can be provided for access by third parties.
  • a feature of the invention is the ability for groups of users to collaborate 4135, even over diverse geography and time.
  • a first user could set up the question to be explored and another user or team could provide the individual models, such as proteins, nucleic acids, or crystals.
  • a different set of people could provide interesting modeling algorithms and the entire collaborative process could even feedback on itself, with different parts of the team making changes along the way based on extrinsic and intrinsic insights.
  • FIG. 42 is a diagram for planning a simulation.
  • the system to be modeled is defined, addressing such parameters as the involved molecules, the environment, and scale.
  • the system can determine whether an all-atom simulation is compatible with the scale. For example, an all- atom representation of a year in the life of a human is probably not compatible with
  • FIG. 43 illustrates a workflow for creating a coarse-grained (CG) molecular dynamics (MD) simulation. Once the system is defined, each molecule type is parameterized to generate a topology. The molecular topologies are gathered by the system and an initial system
  • the system may optionally be minimized (e.g., preventing steric clashes or high-energy configurations or performing heuristic optimizations) or equilibriated (allowing the molecules to adapt to their environments). The simulation is then run.
  • FIG. 44 shows a workflow for parameterization of novel coarse-grained models. Once any atoms are mapped to coarse-graining and bond distributions are measured, bonded terms are created, and a CG simulation is run. If the simulation does not crash and the bonded distributions match, the experimental properties are calculated. If the properties match, the simulation is output.
  • FIG. 45 gives a workflow for parameterization of novel molecules from an ab initio quantum simulation following essentially similar logic.
  • workflows can be performed by the system based on collaborative input from multiple users. For example, a plurality of users can collaboratively work and influence the entire process— from assembly of the model, rigging, simulation and visualization process. By such means, a multi-disciplinary team can contribute to a complex task. Shared collaboration according to the invention also catalyzes communication between team members who may not otherwise effectively
  • FIG. 46 summarizes current approaches to simulation and modeling for drug discovery and development and compares those approaches to methods of the invention.
  • target identification and validation can take years, after which additional years are spent on hit generation. Each step is often left to different teams. The results of hit generation are passed off to lead optimization and after years there may enter pre-clinical.
  • the depicted current approach suffers by lacking a unifying underlying model structure to which each user collaboratively contributes.
  • systems and methods of the invention provide a BlOmolecular Design Ecosystem ("BIODE").
  • Important functions included in the design ecosystem include model building, simulation, and visualization. Model building is provided by the system with integration and ease of use via a central interface. Inputs from different collaboration teams are integrated as input into the underlying model or simulation. Thus the simulation is integrated with the modeling interface. As inputs are given, a simulation can be run and inputs can be adjusted by the users as needed. The simulation can be provided for viewing as a visualization.
  • a chimeric antigen receptor is an engineered receptor in which some arbitrary specificity is grafted to an immune effector cell.
  • CAR chimeric antigen receptor
  • One potential use of a CAR involves removing T-cells from a cancer patient, modifying the T- cells to express receptors specific to the cancer (thus making them into CARs), and reintroducing those CARs into the patient.
  • a likely and important question in developing a CAR is how (and by what mechanism) does varying the "spacer" improve the CAR potency?
  • users can model CAR targets, model CAR candidates, and perform a simulation for the discovery of optimal CAR leads.
  • Modeling CAR targets such as carcinoembryonic antigen (CEA) may start with a curated model that includes the target amino acid sequence. Available structural data is mapped against the sequence and structure is finalized. A simulation is planned according to the diagram in FIG. 42.
  • CEA carcinoembryonic antigen
  • Modeling CAR candidates may include modeling components such as a single-chain variable fragment (scFV) targeting fragment, a spacer, and a transmembrane domain (TM). Similar to modeling the CAR targets, this can include starting with a curated model that includes the candidate amino acid sequences and structural mapping. To explore the effect of varying the spacer, spacer domains are varied and the best initial structures are finalized.
  • scFV single-chain variable fragment
  • TM transmembrane domain
  • model can refer to the model of the entire system and that within the system various components may each independent be provided by a model, e.g., a rigged model as defined below.
  • a model e.g., a rigged model as defined below.
  • the selected molecular dynamics modality is implemented.
  • an environment is included. For example, a pre-set environment may be selected (e.g., from a menu). Parameters are set (e.g., simulation time and length scales). In the embodiment diagrammed in FIG.
  • the selected molecular dynamics modality includes performing a molecular dynamics simulation and a coarse-grained simulation.
  • Molecular dynamics relates to a simulation of physical movements of atoms and molecules in the context of N-body simulation. The atoms and molecules are allowed to interact for a period of time, giving a view of the motion of the atoms.
  • the trajectories of atoms and molecules are determined by numerically solving the Newton's equations of motion for a system of interacting particles, where forces between the particles and potential energy are defined by interatomic potentials or molecular mechanics force fields.
  • CG coarse graining
  • CG-DMD discontinuous molecular dynamics
  • Go-models Go-models.
  • Coarse-graining is done sometimes taking larger pseudo-atoms.
  • Such united atom approximations have been used in MD simulations of biological membranes.
  • the simulation provides for the in silico discovery of phenomena that will arise from the interaction of the modeled actors.
  • optimal CAR leads may be discovered.
  • a team of users may thereby determine what spacer length or type optimizes the function of CAR potency.
  • Systems of the invention operate to pull together the relevant elements for modeling and also build the model, the molecular dynamics modality, and any selected environment.
  • FIG. 47 illustrates components of a CAR molecule.
  • the spacer is provided by CEA moieties that operate to disposes the MFE23 binding site with respect to the cell membrane, in the intercellular space.
  • Systems of the invention can build this model by pulling relevant elements from an underlying curated model database, discussed in more detail below.
  • FIG. 48 illustrates a table breaking out components of the CAR molecule.
  • Systems of the invention can build the table "behind the scenes" for using in building the model or could optionally make such a table available for user collaboration and interaction.
  • a user could view such a table and edit components (e.g., could substitute a different protein databank (PDB) ID # for one or any of the CEA domains).
  • a plurality of users may contribute to the simulation by selecting one or more molecular dynamics modalities or digital models for inclusion in the simulation.
  • Structures, agents, and phenomena that may be modeled using systems and methods of the invention include any suitable ones of those discussed in Whitford, 2005, Proteins: Structure and Funtion, Wiley 542 pages; Alberts and Johnson, 2014, Molecular Biology of the Cell, Garland Science, 1464 pages; Green and Sambrook, 2012, Molecular Cloning: a Laboratory Manual (Fourth Edition): Three-volume set, Cold Spring Harbor Laboratory Press, 2028 pages, the contents of each of which are incorporated by reference.
  • the system may offer relevant simulation methods and do so in a way that alerts the user to the computational feasibility and requirements (i.e. if you are using this model and want to simulate in a water box, you should probably use a Brownian Dynamics simulation using the Martini coarse-grained water model).
  • FIG. 49 shows a graphical interface 129 by which a user may select a model, a molecular dynamics modality, or an environment and build the simulation.
  • systems of the invention can help users in the design of novel molecular entities by offering a function-based menu of molecular domains. For example, if a user is designing a cytoplasmic protein and needs to properly localize that protein to a membrane for the protein to function properly, the user could select a 'membrane recruitment' or 'membrane localization' tab of the GUI.
  • the GUI could include a molecular domains/parts menu and a number of relevant structures/rigs could be provided there, ready for modeling and appending to the rest of the molecule being designed (for example, like PH or C protein domains, or a farnesyl chemical group).
  • FIG. 50 diagrams a system that can be used to provide a biomolecular design ecosystem (BIODE) according to certain embodiments.
  • the system preferably includes a server system 5001 operable to communicate with one or any number of user devices 125.
  • a user device 125 can show a simulation 5051 through interface 129.
  • the server system 5001 preferably includes a processor 5015 coupled to storage, which preferably includes a model database 109.
  • the system may also include an asset database 105.
  • the model database 109 includes curated, rigged models 121.
  • the databases 109, 105 and the rigged models 121 are discussed in greater detail below.
  • the server system 5001 additionally includes at least one user- selectable molecular dynamics modality 5009.
  • the user selectable molecular dynamics modalities 5009 include Brownian, Monte Carlo simulation (MC), explicit- solvent coarse graining, implicit- solvent coarse graining, all-atom simulation, quantum mechanical simulations, hybrids, others, or combinations thereof. It is additionally noted that these need not be mutually-exclusive categories. Any one of the molecular dynamics modalities may overlap with, or be a subset of, any of the others, yet each may still be offered as an optional molecular dynamics modality for the user to select.
  • the server system 5001 may include a module embodiment the planning simulation steps disclosed in FIG. 42 to aid a user in selecting the appropriate molecular dynamics modality.
  • the processor 5015 may parameterize the model according to the workflow set out in FIG. 43 or in FIG. 44. Additionally, the server system 5001 may include a user selectable environment 5023 such as a cell membrane, cytosolic environment, or other.
  • the system 5001 preferably includes pre-modeled or pre-populated (but still
  • a model can be simulated. This may be used to provide a 'panel' of controls against which to simulate novel molecular entities. For example, if a new protein therapeutic has been modeled, it can be examined for binding to a receptor on a surface chosen by user (such as a colon cancer cell plasma membrane). However, the new protein therapeutic could further be simulated against a panel of other common cellular (and inorganic surfaces such as those found on medical devices) to model and predict any adverse interactions. In other words, a researcher could probe such questions as "is this protein sticking to the surface of normal colon epithelium as well?” or "is this protein interacting with artificial surfaces like PLGA?”
  • Systems and methods of the invention may be used to integrate multiple steps of what is currently experienced as a discontinuous process of modeling, simulation, and visualization.
  • FIG. 51 illustrates a simulation 5051 being provided as a visualization through an interface 129 of an electronic device 125.
  • the invention facilitates use by a new type of user (e.g., users who wouldn't otherwise know how to do this type of work) thereby democratizing the use of modeling and simulation in molecular/cell biology. Since the system includes built-in menus for parameterizing simulations, the invention overcomes prior-art problems in which knowing which simulations to run and how to parameterize them is not trivial to the uninitiated. Thus systems and methods of the invention provide the power and advantage of integration and presets at all stages— modeling, simulation and visualization.
  • system 5001 may be operated to receive a new digital model 121 from the user and add the model 121 to database 109.
  • System 5001 may then create a simulation using an optional environment model, the new digital model, and a user-selected molecular dynamics modality.
  • the simulation may, for example, include a number of the digital models in a pathway animation depicting a cascade of events in which at least two depicted biological structures interact only indirectly.
  • the selected molecular dynamics modality allows the user to control one or more of— for example— temperature, salinity, pH, osmolality, or viscosity.
  • a user may create the simulation and share a visual version of the simulation with a plurality of viewers.
  • BIODE biomolecular design ecosystem
  • a visualization product may include a single digital asset or a plurality of digital assets.
  • Exemplary digital assets include pictures, animations, interactives, simulations, games, and other media.
  • Exemplary visualization products include, without limit, electronic textbooks, animated simulations of biological phenomena, educational games, and high quality illustrations.
  • the visualization product provides a visual narrative of a scientific concept without the assistance of text to link together separate digital assets of the visualization product.
  • the visualization product operates to convey a scientific concept to an audience without the use of any text.
  • Visual assets are built of curated models, discussed in greater detail below. Additionally, the system offers embedded assessment, which evaluates the user's retention of concepts covered.
  • FIG. 1 generally illustrates an architecture of systems of the invention.
  • Systems of the invention can be used to provide visual scientific content for example, as customized assets, animations, stills, interactive "decision-tree" visualizations, curricula, and immersive learning environments.
  • the visualization product may be a single digital asset or a plurality of digital assets.
  • the provided visual products may be described as curricula or animations, for example, but it can be understood that a plurality of digital assets is one general form of such a visual product.
  • a visualization product may generally include a grouping of digital assets that explain a particular topic.
  • One important type of visual product is a mini-curriculum.
  • a mini-curriculum may refer to an organized group of digital assets supplemented with educator support materials, assessment materials, and other materials for use in the classroom or other educational context.
  • a visualization product that includes a plurality of digital assets may be made by drawing on an asset database 105.
  • Digital assets within asset database 105 generally refer to an image, an animation, an interactive diagram, a mini-game, or such a piece of digital media.
  • a digital asset will include one or more curated models from a curated model database 109.
  • the present invention generally relates to curated model database 109.
  • Curated database 109 generally includes one or a plurality of rigged curated models 121.
  • a curated model may generally be understood to refer to a 3D model of a molecule, organ, organisms, instrument or other that is constructed from multiple data sources (such as structural, dynamic and other sources) and rigged so as to be 'scene-ready' for production.
  • a curated model may also include embedded within all the sources and techniques used in the modeling/rigging (and other curation) activities.
  • a curated model includes a multi-dimensional (e.g., 3D molecular) model that integrates scientific information (structural, dynamic, and other) that is 'ready to use' for visualization.
  • Curated models 121 may be built de novo or by sourcing scientific data from a suitable source such as, for example, a simulation, structural data (e.g., from protein data bank), dynamic data, or the scientific literature. Curation includes selection or building of a model and rigging or simulating the model to produce a rigged or posed model 121. Rigging or simulating a model can make a model 'ready to use' for visualization. It is noted that a user for the curated model database may be a scientific animator (when models from the database are imported into a 3D app like Maya and then used to create a visualization, static or dynamic).
  • One novel feature of a curated models database includes the way in which the models are accompanied by data which may specify (i) what pieces of a model were derived from what kind of data (X-ray vs. NMR vs. cryo-EM vs. modeled de novo using hypothetical data vs.
  • the model for a transmembrane protein may include— besides the structural data itself such as the shape(s) of the protein and its known range of motion—the transmembrane domain being flagged with metadata such that the protein embeds itself properly into a lipid bilayer when combined with a model or simulation of a lipid bilayer membrane.
  • Another kind of data includes sites of post-translational modifications such as phosphorylation, glycosylation, or others.
  • Components of system 101 may be interacted with by a variety of different users. Non- limiting examples of users with respect to FIG. 1 are given.
  • the simulations, structural and dynamic data and primary literature that feed into model database 109 may be used by scientists or system administrators to make curated models 121.
  • An animator may use model database 109 and the curated models 121 to create digital assets that populate asset database 105.
  • a teacher, publisher, or content provider may use the digital assets to create custom animations, stills, mini- curricula, collections, and other media. End-users such as students, scientists, or the consumer public may use the custom animations, stills, mini-curricula, collections, and other media. It will be appreciated that any of the users or others may use any element of system 101.
  • FIG. 2 gives an exemplary structure of a rigged model 121.
  • a rigged model 121 will generally include a model 203 and a rig 207.
  • a rig is known in the art of 3D animation and generally refers to a 3D construct that provides an organized system of deformers, expressions, and controls applied to a model and that specifies and drives the motion of the model so that it can be effectively animated or simulated.
  • a rig may include joints, bones, particles, springs, or other concepts. Rig has been used in the animation arts to include a deformation engine that specifies how movement of a model should translate into animation of a depicted entity based on the model.
  • a rig provides software and data used to deform or transform a neutral pose of a model into a specific active pose variations.
  • animation software manipulate a rig incorporated to a model, animated or simulated movement of the model is achieved.
  • Rigging may sometimes be referred to as character setup or animation setup.
  • a detailed discussion of creating rigs may be found in sources such as O 'Hailey, 2013, Rig it Right! Maya Animation and Rigging Concepts, Focal Press, Burlington MA, 280 pages; Palamar and Keller, 2011, Mastering Autodesk Maya 2012, Wiley Publishing, 950 pages (esp.
  • Model 203 includes data representing a structure, often in the form of a geometry file or particle cloud/object. Any suitable model 203 may be included in a rigged model 121.
  • the model may represent a single molecule, an assembly of molecules, or a structure or systems. Examples of things that may be represented by a model include a protein, a nucleotide, a polymerase bound to a strand of DNA, a solar system, a skeleton, a machine, or others.
  • model 203 is a geometry or particle object file(s) of a format suitable for creation, viewing, and manipulation within modeling or animation software such as, for example, Autodesk Maya. Any suitable animation software may be used.
  • Exemplary animation software products include those provided by Cinema4D Studio by Maxon Computer Inc. (Newbury Park, CA), Blender supported by the Stichting Blender Foundation (Amsterdam, the Netherlands), and 3DS Max 2014 by Autodesk, Inc. (San Rafael, CA).
  • any suitable method may be used to obtain a geometry or particle file.
  • the information necessary to create geometry or particle files can be imported from sources such as structure database, created de novo within a modeling environment, or built of raw data obtained from an experiment or assay.
  • the structures to be represented by geometry or particle files may be predicted by computational algorithms, or may represent real structures determined by spectroscopic methods such as X-ray crystallography or nuclear magnetic resonance (NMR).
  • One exemplary approach to obtaining geometry files includes the use of a molecular graphics application such as Chimera or PyMOL.
  • Suitable applications may include Astex Viewer, UGENE, DS Visualizer, Swiss PDB Viewer, Interchem, VMD, RasMol, Jmol, Python Molecular Viewer, Coot, MDL Chime, MolSoft Viewer, and other such products.
  • Such a program can be used to open raw structural data, such as a set of coordinates from a protein databank (PDB) file and to export the structural data in a format suitable for use in a modeling environment.
  • Raw structural data can also be used to generate a particle file for use in a modeling, animation or simulation environment.
  • a PDB file embodies a format for representing actual 3D structures of biological molecules.
  • the PDB format is widely accepted as a standard in the biosciences.
  • the molecules may include protein, nucleic acid (RNA or DNA), lipids, carbohydrates, other molecules or macromolecules, a complex of several proteins, a complex of protein with nucleic acid, or any combination thereof including but not limited to these in a complex with small molecule ligands such as drugs, cofactors, metal ions, etc.
  • the 3D structure of the macromolecule is usually determined by X-ray crystallography, but other spectroscopic methods, such as NMR, or microscopic methods, such as cryoEM, are occasionally employed.
  • the Protein Data Bank currently archives close to 100,000 PDB files of molecular structures, which are freely available to the public. See, e.g., Berman, et al., 2000, The Protein Data Bank, Nucl Acids Res 28(1):235- 242.
  • the PDB format includes ASCII text giving XYZ coordinates for atom locations, as well as data on atom-to-atom bond connections. Other information typically included are protein amino acid sequence and secondary structure, crystallographic space group, and general comments on the biological role of the protein. Molecular graphics applications such as Chimera or PyMOL by design readily import PDB files.
  • the structural data can be exported from the molecular graphics application (e.g.,
  • Chimera, PyMOL Chimera, PyMOL to generate geometry files.
  • These may be exported as Virtual Reality Modeling Language (VRML) and then converted to OBJ format (a common data format for 3D data) before being imported into a modeling program such as Maya. Additionally or
  • scripts can be used to prepare a geometry file from a set of coordinates using, for example, Maya Embedded Language (MEL).
  • MEL Maya Embedded Language
  • the method to use may relate to what will be done with the geometry once inside Maya.
  • large PDB datasets are brought into Maya as geometry files using the multi- scale model feature of Chimera.
  • structural data can be obtained for modeling using products like the Molecular Maya Toolkit, sometimes referred to as m Maya or Molecular Maya, the embedded Python Molecular Viewer, sometimes referred to as ePMV or BioBlender.
  • Molecular Maya is a free software toolkit that extends the capabilities of Maya by allowing users to import, build, and animate molecular structures.
  • Molecular Maya includes the functionality to open PDB- and other formatted files.
  • Molecular Maya works with Maya 2011, 2012, 2013, and 2014 and adds a molecule- shaped icon to the Maya environment.
  • Molecular Maya includes (or adds to Maya) UI elements for opening PDB files.
  • Molecular Maya can import the text-formatted native PDB file.
  • Maya or Molecular Maya
  • Molecular Maya will present an empty scene upon opening. Once a PDB file is imported, it can be viewed as atoms. However, Molecular Maya can transform it into a geometric or particle structure, with options for selecting levels of resolution. Once imported, the geometry and/or particle file provides the model 203 for a rigged model 121. Molecular Maya allows a curator to import a range of structural pieces which may then be assembled by hand (or simulated) to create model 203 or a model 121.
  • FIG. 3 diagrams a method 301 for providing a curated model.
  • Method 301 operates preferably within the context of determining an objective 321. For example, it may be the objective to provide a curated model of biological macromolecules.
  • a computer system is used to obtain 327 structure data. Structure data can be obtained from a scientific assay such as x-ray crystallography, either directly or once published (e.g., from PDB files) or a combination of these.
  • the structure is used to build 331 a model 203, typically a geometry or particle file.
  • the model may optionally be surfaced 335 with textures or shaders.
  • the geometry is rigged 339 with a rig that defines animation dynamics for the structure such that a range of motion for the rigged model is defined (i.e., for the depiction of the underlying structure in a downstream animation).
  • Each curated model is accessioned 343 to curated model database 109. Access to these rigged, digital models 121 is then provided for use in illustrating scientific concepts. Access may be provided through, for example, asset database 105, in which one or more rigged model 121 may be bundled into digital assets.
  • a curated model database 109 may include sets of models that are tailored to illustrate biological systems or concepts.
  • a curated model database 109 includes models to represent all of the components of a cell.
  • the database includes entries for each of the components of a cell such as, for example, all of the structures that make up the membrane, cytoplasm, and nucleic acids, as well as a variety of proteins, lipids, and carbohydrates, in all cells.
  • Database 109 may be further tailored to provide curated models 121 for representing specific cell types (e.g., eukaryotic or bacterial).
  • a eukaryotic cell database may include structures for the nucleus, chromosomes, ribosomes, microtubules, microfilaments, centrioles, cilia, flagella, and other structures.
  • An animal-based database 109 may include organelles such as the nucleus, the nucleolus (within the nucleus), rough and smooth endoplasmic reticulum, Golgi apparatus, mitochondria, vesicles, lysosomes, centrosomes, centrioles and other such structures.
  • a plant cell database 109 may include entries for the cellulose cell wall, central vacuole, and chloroplasts, as well as organelles.
  • Fungal cells may include chitinous cell walls.
  • a bacterial cell curated model database 109 may include models representing the cell wall (e.g., thick peptidoglycan for Gram + bacteria), plasma membrane, extracellular structures such as fimbriae and pili, S-layers, glycocalyx, and flagella.
  • Intracellular bacterial components include the bacterial chromosome and plasmids ribosomes and other multi-protein complexes, intracellular membranes, cytoskeleton, as well as nutrient storage structures such as inclusions, vacuoles, or other micro- compartments.
  • Archaea cells may have a lipid monolayer membrane.
  • FIG. 4 illustrates components of a computer system 401 that may be included in systems of the invention.
  • asset database 105 operates with the ability to connect to and pull from curated model database 109.
  • a curation computer device 407 is used to create curated models and populate model database 109.
  • Computer device 409 is used to build digital assets that include the rigged models 121.
  • curation computer device 407 and computer device 409 are being described in terms of their roles. These roles can each separately be performed by using one or any number of different computers and can even both be performed through the use of a single computer.
  • a computer generally refers to a device that includes a processor coupled to a non-transitory memory and an input output device.
  • Computers of system 401 may communicate with one other via a network— broadly referring to the hardware used in transferring signals between computers.
  • Network 401 may be taken to include internet hardware such as telephone lines, cell towers, local switches and routers (e.g., LINKSYS products by Cisco Systems, Inc. (San Jose, CA), Ethernet cables, Wi-Fi cards, network interface cards, and other such device.
  • Network 401 may be understood as providing the ability to obtain structures from a structure database such as, for example, protein databank.
  • system 401 provides a construction computer device 423 for constructing a visual product (which device may be provided by one or more separate, dedicated devices or may be provided by the same one or more computer device providing either or both of curation computer device 407 and computer device 409).
  • curation computer device 407 and computer device 409 are employed in a production environment, wherein skilled scientist-animators rig models and build assets.
  • curation method 301 is performed using curation computer device 407.
  • the output of curation method 301 will include at least one rigged model 121.
  • construction computer device 423 refers to the personal computer (e.g., tablet, laptop, or desktop) used by a consumer to log into system 401 and order, design, or put together a visualization project to communicate a scientific idea.
  • a visualization product may include one or more of a picture, an animation, a simulation, a game, an interactive model, or other such media. Components of animations, simulations, and other interactive media can operate based on animation principles. Models such as PDB-based structures can be rigged and animated using animation and modeling software tools.
  • FIG. 5 shows a modeling and animation tool as presented by system 401, i.e., a screenshot from a modeling and animation environment (e.g., as implemented on curation computer device 407).
  • the animation environment is provided by
  • Molecular Maya (e.g., m Maya v 1.0 or future versions) and is used to rig a model such as may be obtained from a PDB file.
  • System 401 can be used to create models, rig models, simulate models and create visual products such as animations that use those models.
  • a model— geometry or particle file— can be rigged, and the rig will typically include a reference to the geometry file, it will be appreciated that a rig can be changed to reference a different model. That is, one of the valuable properties of a rig is that it can be used with one geometry then another. For example, a modeler could make a "quick and dirty" geometry and hand it off to the rigger. The rigger could build a rig using that geometry while the modeler works on a more detailed geometry. However, as used within an animation, a rig will generally reference one model (i.e., the geometry that it rigs).
  • system 401 includes Maya and models 203 are represented through the use of Maya's dependency graph.
  • Maya is one example of an environment useful here, but there are others and the models that live in the curated model database can be created in 3D software environments other than Maya.
  • Geometric objects, as well as data processing units such as transforms and shaders, are encapsulated as nodes. These nodes are connected through their attributes into a network that is known as the dependency graph. Each node is dependent upon another, which includes that as the dependency graph is dynamically updated, changes to any node automatically propagate through the graph to all other nodes which are dependent on it. This dynamic updating of the dependency graph is the core of the real-time graphics engine of Maya.
  • FIG. 6 shows a protein model 601 according to certain embodiments.
  • each polygon or non-uniform rotation b-spline (NURBS) curve of model 601 may be included as a node in the dependency graph.
  • a complex geometry such as model 601 can be obtained by building within Maya or by import.
  • PDB files can be imported directly into Molecular Maya or by exporting VMRL from a molecular viewer.
  • complex geometries can be built within Maya using tools for 3D modeling.
  • a 3D model includes the geometry provided by surface. Maya supports three surface types: polygons, NURBS, and subdivisions.
  • a polygon geometry includes a surface made up of polygon faces with shared edges and vertices. Polygonal surfaces can be split, removed, extruded, and smoothed.
  • One of skill in the art of 3D modeling will recognize the great breadth of geometries that can be created with polygon surface. So too with NURBS geometries, which basically comprises surfaces created over a network of NURBS curves and converted to triangles when rendered. Subdivision surfaces, or subDs, are a way of adding detail to particular sections of a mesh by subdividing the existing surfaces.
  • a 3D model may also be created with particles. This particle object can either remain particles or be used, in turn, to generate surface geometry such as an isosurface.
  • FIG. 7 depicts a model 701 representing a reovirus sigmal protein.
  • Reovirus attaches to cellular receptors with the sigmal protein, a fiber- like molecule protruding from the 12 vertices of the icosahedral virion.
  • the receptor-binding fragment of sigmal includes an elongated trimer with two domains: a compact head with a beta-barrel fold and a fibrous tail containing a triple beta-spiral. See Chappell, et al., 2002, Crystal structure of reovirus attachment protein sigmal reveals evolutionary relationship to adenovirus fiber, EMBO J 21: 1-11.
  • Model 701 can be made by any suitable method such as, for example, approximating the outer surface of the molecule by drawing a NURBS curve and rotating it around the Y axis.
  • model 701 is made by importing data from a PDB file, specifically from PDB # 1KKE.
  • a PDB file can be imported directly into a program such as molecular Maya or a PDB file can be opened in a viewer (e.g., PyMOL) and exported as VRML which can then be opened by a program such as Maya or Molecular Maya to arrive at model 701 as shown in FIG. 7.
  • FIG. 8 illustrates rigging model 701.
  • Rigging includes the creation of organized systems of deformers, expressions, and controls applied to an object so that it can be animated well.
  • a rig will allow an animator to create an animation without himself doing the rigging. That is, rigging is uncoupled from animation or simulation, allowing different tasks to be performed by specialists.
  • rigging is a continuously evolving practice. Typically, rigging will include starting with a geometry or particle object such as model 701, building a skeleton, creating the rig and weighting the geometry.
  • the skeleton is built by adding joints 805 to model 701.
  • Rigging can include using Maya's Joint Tool from the Animation menu to create a skeleton when, for example, beginning work on a geometric structure.
  • the Joint Tool can be used to introduce joints into the mesh, which will be connected by bones (here, bones, joints, and skin refer to the control tools known in the animation arts).
  • Joints are oriented in that their axis (e.g., defining the pivot) is oriented appropriately. Typically, orienting is done before the geometry is bound to the skeleton. In Maya, a joint will be represented by a wireframe sphere.
  • a skeleton may be bound to a skin so that, when bones and joints of a skeleton move (e.g., according to inputs and a rig), the skin presents a visible surface that deforms (e.g., according to how it is bound to the skeleton).
  • first joint 805a is created at the end of the fibrous tail of the monomer.
  • second joint 805b is created, bone 809a is created extending from first joint 805a to second joint 805b. This process is continued for all of model 701.
  • FIG. 9 shows model 701 with a set of joints 805 connected by bones 809 and a dialog box 901 for binding model 701.
  • FIG. 9 illustrates skinning the geometry of model 701. Skinning geometry is the process in which geometry is bound to joints so that, as the joints are rotated or translated, the geometry is deformed. The terms skinning and binding are generally
  • FIG. 10 shows a motion of a model 701 based on the applied rigging.
  • the sigmal monomer has bent around joint 805.
  • Rotation around joints can be controlled by kinematic concepts, as provided for within animation environments such as Maya and molecular Maya.
  • Such animation environments provide for controls such as forward kinematic and inverse kinematic controls of systems of joints.
  • Forward kinematics refers to having each joint in a chain inherit the motion of its parent joint
  • inverse kinematics IK refers to causing joints to orient themselves based on the location of a goal known as an end effector.
  • an amino acid side chain in the active site of an enzyme may be rigged with inverse kinematics using the substrate as the end effector.
  • a protein subunit that undergoes a tertiary structure reorganization while changing conformations may be modeled using forward kinematics.
  • animation involves the use of deformers such as blend shapes.
  • a blend shape deformer allows a depicted structure to morph between two meshes and allows a user to control the blend and the morph.
  • at least two topologically identical meshes are created, representing the structure in at least two corresponding conformations.
  • a blend shape is created from the meshes and a node network is created that will work with constraints and rig controls to adjust the animated transformation between the two conformations.
  • the two meshes are selected and the Blendshape command is run from the Create Deformers menu.
  • a new node is created and one of the meshes can be deleted (now being represented by the Blendshape).
  • a rigged model includes an animation rig that is easy to understand.
  • controls are labeled and easy to select.
  • entering 0 in the translation channels for the controls return the rig to the start position.
  • IK handles use world space coordinates so setting translation channels to 0 moves the handle to origin.
  • systems of the invention may be operable to register and warn against impending self-intersections through the use of self-aware rigging techniques applicable to scientific structures such as biological macromolecules.
  • collision detection rigging can include the use of electrostatic forces (e.g., as mapped to the surface of a space-filling model).
  • electrostatic forces e.g., as mapped to the surface of a space-filling model.
  • collision-detection rigging i.e., abiding by electrostatic concepts providing that like-charged surfaces repel and unlike-charges attract
  • the one or a set of MEL scripts not only create Maya-native geometry directly from the PDB but also automatically create a rig that has some inherent motion constraints applied.
  • the automatic rigging may be applied with different types of molecular representation (ball & stick versus cartoon for example would have very different 'rules' applied to constrain motion).
  • a MEL script can apply certain rigging to certain structural motifs automatically and by default. For example, the peptide bonds of a polypeptide can be automatically rigged for realistic rotations. The rigged model can be provided for "fine tuning" by a user by hand.
  • information for the rig is obtained from a scientific data source.
  • the conformational dynamics data bank (CDDB) can be accessed to obtain information about possible conformations of a protein.
  • a rig can be created to restrict the range of motion of the protein model to conformations allowed by the conformation data bank information.
  • a MEL script can be used to automatically create that rig and apply it to the model based on CDDB data.
  • the CDDB is described in Kim, et al, 2011, Nucl Ac Res 29:D451-5. Suitable databases for protein dynamics may be discussed in Liu & Karimi, 2007, High- throughput modeling and analysis of protein structural dynamics, Brief Bioinform 8(6):432-45.
  • curated models of the invention are suited for employment in modern gaming engines.
  • the digital assets (models, textures, rigs) used to develop high- end games are created in packages like Maya.
  • molecular-movie style animations are generated within an environment such as Maya for application within interactive molecular environments for educational purposes.
  • embodiments of the invention can use rigging concepts to depict motion through animation and can even be used to control levels of granularity at which motion can be depicted. For example, at one level, the overall motion of molecular structures within their environments can be shown, while at another level, motions at the atomic level can be depicted.
  • FIG. 11 illustrates protein dynamics at four different levels that can be illustrated using modeling and rigging concepts discussed herein.
  • the diffusion or random motion of entire proteins can be illustrated.
  • conformational changes associated with domains of proteins can be depicted, for example, within an animation provided by methods of the invention involving the use of rigged models.
  • the various side chain rotations of individual amino acids can be depicted.
  • the thermal vibrations of individual atoms can be depicted.
  • process such as diffusion and random motion or Brownian motion can be modeled as stochastic process and such processes can be implemented using computer programming or scripting.
  • MEL or Python scripting may be employed.
  • MEL or Python scripts start directly from a PDB coordinate file and generate ribbon, surface or particle representations. In some embodiments, the MEL or
  • Python scripts read from the PDB file, e.g., atom-by-atom. Typically, a set of coordinates will be given to each atom and any bonds indicated in the PDB file will be treated as indicating a connection to another atom. Shading groups are created in the Maya dependency graph. MEL scripts set shading for each atom and create a sphere in the dependency graph. For each bond, a cylinder is created. These models created by MEL scripts may be lighter and cleaner that exports from Chimera or PyMOL since they have been built within Maya using optimized types of geometry, such as NURBS, for example. The geometry file once loaded into Maya appears as a structure in a display.
  • the protein molecule will be displayed (see FIG. 6 for an example).
  • the molecule in the display can be rotated, translated, and scaled using Maya's native functionality (e.g., hold down ALT + L, M, or R mouse button, respectively, while dragging) for transforming the scene view.
  • a molecule may be displayed using a known format such as a ball and stick model. Sticks represent bonds and balls represent atoms.
  • a molecule may be displayed using a surface model— i.e., showing a surface of the molecule.
  • methods of the invention are implemented by programming within an animation environment. Besides MEL and Python, Maya provides an application
  • each model 121 is accessioned 271 to curated model database 109. Access to these rigged, digital models 121 is then provided for use in illustrating scientific concepts. The following discussion describes potential uses of database 109.
  • FIG. 12 diagrams a method 1201 of making a visualization product that includes a plurality of digital assets that can be used for conveying a scientific concept.
  • the conveyance of some scientific concept is determined 1205 to be an important objective.
  • Any suitable scientific concept can be conveyed using methods of the invention including, for example: protein structure, dynamics, activity, or binding; embryonic development; cosmological concepts or general relativity; cell biology phenomenon such as cell signaling pathways or actin/myosin function; organismal behavior such as eusocial insect colony function; chemical phenomenon such as small molecule effects on targets; biochemistry and metabolism; molecular biology; and others.
  • a storyboard is developed 1209 that will determine a digital asset to be produced.
  • Developing 1209 a storyboard may merely refer to ordaining a structure by which material will be presented. For example, developing a storyboard may consist only of deciding to obtain a still image, or developing a storyboard may include planning a series of scenes for inclusion in a complex, multi-actor animation (e.g., showing TATA binding and recruitment).
  • rigged models 121 are obtained 1213 from database 105 for inclusion in a digital asset.
  • the rigged model(s) 121 to be included will relate to the natural phenomenon to be represented. For example, protein conformations can be illustrated by using a rig and a protein model that work together to illustrate the protein assuming a plurality of realistic conformations within an animation.
  • method 1201 may be used to depict complex phenomenon.
  • Rigged models 121 included in digital assets may be found to be particularly valuable for illustrated concepts that some students struggle with.
  • a digital asset can be a pathway animation depicting a cascade of events in which at least two depicted biological structures interact only indirectly.
  • MAP kinase cascade may be well illustrated using a digital asset that includes an animation. Due to the nature of character rigging, indirect interactions can be understood.
  • MAP kinase kinases aka MAP2 kinases
  • upstream kinases e.g., MAP3 kinases
  • MAP3Ks such as c-Raf, MEKK4 or MLK3 themselves require multiple steps for activation.
  • MAP kinases exist that phosphorylate serine or threonine residues near proline on cytosolic proteins and also phosphorylate transcription factors during transcription.
  • an animation can illustrate the indirect interactions between, for example, c-Raf and a classical MAP kinase such as ERK1. Since each protein (c-Raf, a MAP2K, ERK1) is included with a structurally accurate model 203 and a dynamically accurate rig 207, an audience can view the indirect influence of c-Raf on transcription via an animation that is scientifically accurate. Additionally, this material can be illustrated through, for example, a web-based interactive decision tree, allowing a Georgia student to select input and decide conditions that control a depicted outcome. As discussed, a digital asset can include an animation.
  • a digital asset could be, for example, a still, a simulation, an interaction, a game, or other media.
  • One or a number of high-quality still images that depict natural phenomena with scientific accuracy may be desired by a publisher.
  • Stills can be composed using models from database 109. For example, if a publisher wishes to illustrate the so-called central dogma of molecular biology to a high-school audience, systems and methods of the invention can be used to produce three stills, one to illustrate each of replication, transcription, and translation.
  • the nucleic acids and proteins can be included based on models from the database and the images can be stylized to communicate effectively with the high-school education level (e.g., bases can be presented in a simplified structure and each clearly labeled with one of A,T, C, and G).
  • bases can be presented in a simplified structure and each clearly labeled with one of A,T, C, and G.
  • a working researcher may desire a digital asset consisting of a still image illustrating an autocatalytic property of a ribonucleic acid for publication in a peer-reviewed journal.
  • a valence electron cloud for the oxygen of a 2' hydroxyl group that acts as a nucleophile in phosphodiester cleavage can be illustrated and shaded so that readers visualize the ribozyme reaction mechanism.
  • systems and methods of the invention can be used to produce 1217 a visualization product that includes a plurality of digital assets, each digital asset having one of a variety of formats.
  • the digital assets may be tailored to an education level of an audience for the effective conveyance of a scientific concept.
  • Producing a digital asset may include building an animation that uses one or a plurality of rigged models 121.
  • Digital assets are made to be scientifically accurate. This can include, for example, concealing portions or picking alternative geometrical or visual representations for portions of the digital model for which scientific data is not available.
  • digital assets can be tailored to an education level of an audience. For example, a level of complexity of the digital asset can be set according to an education level of an audience that will view an animation. Additionally or alternatively, parts of the digital asset can be concealed based on the education level.
  • systems and methods of the invention are operable to
  • proteins may include information about surface geometry and also information about charge distribution on the surface. If an education level is within K-12, the charge information may be omitted from an animation, whereas if the education level is graduate or higher, the charge information may be included as a color-coded scheme on the surface of individual proteins.
  • Tailoring to an education level can include controlling a number of elements to depict in an animation.
  • an animation depicting transcription initiation if the audience level is set at grade school, systems of the invention may depict only an RNA polymerase processing a DNA strand.
  • the system may include, for example, TATA binding proteins and transcription factors binding and recruiting the polymerase.
  • digital models may include elements or portions that are tagged with an education level so that systems may selectively exclude those elements or portions for education levels that do not match the tag.
  • an education level For example, in biochemistry, it is thought that in an enzyme-catalyzed reaction, the substrate will fleetingly occupy a highest-energy transition state and that the nature of this transition state precludes its ever being observed according to quantum principles.
  • a model of the substrate may include rigging allowing the substrate to assume the transition state form and may further include rigging that vibrates or blurs the surface geometry at the instant the transition state form is assumed to prevent direct and instantaneous
  • transition state For an animation in which the education level is, for example, elementary school, any depiction of the transition state may be excluded and the enzyme-catalyzed reaction may be depicted simply as substrate-in, product out. For college level animations, the transition state may be depicted for an instant during the reaction. For an animation intended for a post-doctoral biochemist with an understanding of quantum physics, the uncertain transition state may be depicted.
  • the digital assets can be manipulated to create scientifically-accurate depictions of natural phenomenon.
  • circumstantial parameters such as temperature, viscosity, salinity, or pH can be set (e.g., some proteins may exhibit different conformations, or some reactions may occur at different speeds, as such parameters vary).
  • a number of proteins are known to respond to [H+] gradients. If, for example, an ATPase is being modeled in a lipid bi-layer membrane, a user may input a hydrogen ion concentration on either side of the membrane. If the concentration is isomolar across the membrane, the ATPase— by virtue of its rigging— will be depicted as static.
  • ATPase will be depicted as active.
  • temperature can be manipulated to influence an animation.
  • an animation environment is set up with rigged models for Taq polymerase, DNA strands, oligonucleotide primers, and dNTPs
  • a user can use an interface provided by systems of the invention to establish a series of different temperatures that will be modeled at different times during the animation. At high temperatures, the DNA will melt, and at cooler temperatures, the oligos will hybridize to the DNA to initiate polymerase activity. By thus setting environmental parameters, a user can successfully model the polymerase chain reaction.
  • Each digital asset is cataloged 1221 by, for example, title, subject matter, client ID, or other information for later retrieval and use.
  • the digital assets are then stored in asset database 105 and used to convey 1225 the scientific concept to an audience.
  • steps of method 1201 can be performed using system 401.
  • system 401 includes a processor coupled to a non-transitory memory having stored therein a plurality of models, each model comprising data representing a structure and a rig that defines animation dynamics for the structure such that a range of motion of each model on an electronic display device 129 is predetermined without manipulation from a user.
  • FIG. 13 diagrams a method 1301 for constructing and providing a visual product using a curated database of the present invention.
  • method 1301 for providing a visualization product includes determining 1305 some science concept or topic to be depicted.
  • a storyboard for the visualization may be developed 1309. No particular format is required for a storyboard.
  • a storyboard generally, is a tool to organize contents of a visual product.
  • digital assets are selected 1313 for inclusion. At least one of the digital assets will be capable of visually conveying at least a portion of the scientific concept. At least one of the digital assets will include a rigged model 121. Use of a rigged model 121 allows models to be animated 1317.
  • a visualization product is constructed 1321 such that it includes at least one digital asset. This visualization product is then provided 1325 for use (e.g., for viewing by the audience on an electronic display device).
  • Method 1301 may include receiving data related to the education level of the audience.
  • the visual product may be any product that visually communicates a scientific concept.
  • the visual product may be an animation depicted on a computer screen or it may include a tangible medium having files stored therein that can be accessed to view an animation.
  • the visual product may include a still photo or an interactive game.
  • the visual product includes a digital textbook (e.g., for viewing via a tablet computer or similar device).
  • Providing the visual product may include rendering an animation (e.g., taking the 3D modeling and animation files and outputting a video clip that comprises a series of bitmapped images).
  • the visualization product will include a set of digital assets that, in the aggregate, convey an entire scientific concept (e.g., protein folding or DNA replication).
  • a visual product is tailored to an education level of an audience. This can include receiving education level information.
  • a customer can order a visualization product (e.g., using a web interface) and may include in the order the information about the audience.
  • Education level can be specified by, for example, grade level, or it can be provided in other terms such as age.
  • the visual product can then be tailored to the grade level. For example, in some embodiments, tailoring the digital asset is done by automatically visually concealing one or more portions of the digital asset based on the data related to the education level of the audience.
  • Constructing and providing a visual product is preferably performed using a system that includes a processor coupled to a non-transitory memory.
  • the system can be used to construct an electronically displayable visualization product that comprises at least one digital asset that visually conveys at least a portion of a scientific concept.
  • the digital asset includes a structure 203 and a rig 207 and is tailored based on an education level of an audience. An end user can access the system to initiate creation of a visual product.
  • FIG. 14 depicts an interface 129 provided by computing device 125 for using systems of the invention.
  • systems and methods of the invention can employ a web front-end or other interface, such as a dedicated application, to allow users to create products described herein.
  • databases, products, and visualizations described herein may include a large number and variety of unforeseen assets or model types (e.g., for things like a cell-type library), it is valuable for the invention to provide an easy-to-use interface for users to put in suggestions or requests. For example, a user can request their favorite cell type or a peroxisome.
  • a user may see a web interface to set up a request for a visualization product that illustrates G-protein coupled receptor activation.
  • any suitable scientific concept may be illustrated by systems and methods of the invention.
  • embryonic development can be illustrated and conveyed by modeling a developing embryo using one or more rigged model 121.
  • one or more entire cell e.g., substantially all components or processes
  • Systems and methods of the invention have particular application to systems that include a stochastic component.
  • it may be illustrative to depict transmembrane proteins as drifting within a lipid bi- layer membrane to communicate the fluidic mosaic model hypothesis of the plasma membrane. See, e.g., Singer & Nicholson, 1972, The fluid mosaic model of the structure of cell membranes, Science 175(4023):720-31.
  • each lipid can be populated to a membrane surface, and each transmembrane protein can be included, using a rigged model 121 for each.
  • a rigged model 121 for each.
  • FIG. 15 shows a method 1501 for creating a visual product according to certain embodiments.
  • Method 1501 includes determining 1505 what product to make such as, for example, a still, a sequence, or an animation.
  • the environment is then constructed 1507.
  • Constructing the environment includes layering 1509 (see FIG. 20) and selecting 1513 layer components.
  • Layer component choices depend on the subject matter, the environment, and the layer. If a cellular biology concept is being communicated, options for components to have within various layers of the visual product may include none, nucleus, plasma membrane exterior, plasma membrane interior, mitochondrion, cytoplasm, others, or a combination thereof.
  • Embodiments of the invention include preset models and user-driven sets of layer components. The components can be customized and positioned 1517. Options are component-specific such as, for example, animation presets.
  • Rendering presets and color palettes are selected 1525.
  • Selecting color palettes can include assigning color by component or using an overall Kuler palette, and can also include using an overall image style (ambient occlusion (AO), simulated electron microscope (EM), cartoon- style, combinations).
  • Ambient occlusion is a method to approximate light shining onto a surface. Typically, ambient occlusion is used for realism.
  • Ambient occlusion models rays cast in every direction from a surface. Rays which reach the background increase the brightness of the surface, whereas a ray that hits an object contributes no illumination. As a result, points surrounded by a large amount of geometry are rendered dark, whereas points with little geometry on the visible hemisphere appear light.
  • Programs such as molecular Maya include shaders such as the EM shader to simulate the appearance of electron microscopy. All of the preceding work can be reviewed 1529, allowing a user to revisit any of the foregoing steps.
  • the product is watermarked 1533. A delivery format is established. The product may then be rendered 1537.
  • systems and methods of the invention can be used to create a variety of digital assets, databases, and visual products.
  • Systems and methods of the invention may include additional features and functionality.
  • a scientific animator my use a curated model to create a digital asset, which could, for example, depict and illustrate such diverse phenomena as polymerization, cell signaling, Brownian motion, lipid bilayer membrane structure, cellular organization, protein folding and conformation, organismal anatomy, embryonic development, bench-top lab experiment protocols, intracellular bio- molecular structure and composition, viral structure and function including capsid packing, the biochemistry of metabolism, phylo genetics, ecological principles, neural function, and other phenomenon.
  • a digital asset may illustrate polymerization.
  • a digital asset may illustrate Brownian motion.
  • a curated model can be used for each of the individual particles (e.g., proteins, molecules, other physical particles), which may exhibit stochastic motion that is illustrated and modeled using the curated models.
  • the audience may be any single person or group of people with any education level, and the invention addresses unmet needs for a variety of different audience types or education levels.
  • the audience may be of a collegiate or post-collegiate level, which may include for example, graduate, medical, post-doctoral or any other level.
  • Content may be provided that is relevant to pre-collegiate, undergraduate, graduate, medical school and post-doctoral.
  • high- school students e.g., in AP Biology
  • a mini curriculum may include, for example, an assessment integrated with curriculum modules in the form of a digital asset as described herein.
  • a module is a singular digital learning asset or "widget”, and can be of a number of different types of media such as static or interactive images or diagrams, interactives, or mini-games, for example.
  • a visual mini-curriculum can be made of a grouping of modules that address traditional curriculum topics.
  • a collection may include a grouping of modules that belong together based on scientific topic, but not necessarily assembled in an education, curricular context like a mini-curriculum.
  • systems and methods of the invention provide for collaborative learning. For example, content may be tailored to support paired, or groups of, students on projects. Material may be delivered such that tasks or response prompts are directed to members of a pair or group to support collaborative learning objectives.
  • a digital, visual mini-curriculum may find value in visual products provided for college students. For example, pre-med students can learn anatomy and physiology concepts. For working research scientists, there is a need for the ability to provide scientifically accurate visualizations in which static or animated visuals are derived from actual datasets.
  • scientists may require a clear provenance of datasets used for a visualization.
  • Visual products as described herein may be used by scientists to illustrate and understand competing models for mechanisms.
  • the general public may be well-served by books, articles, TV shows and documentaries that include scientifically accurate visualizations tailored to the average education level of the general public within a market segment.
  • a society may be better informed and able to bring a fundamental understanding of science to future careers.
  • a mini-curriculum will generally include educational materials and preferably includes tools for assessment.
  • a mini-curriculum of the invention is that, due to the visual nature of the products provided by the invention, the curriculum need not be interwoven with prose exposition as required by convention for existing textbooks and journal articles. While a visual product may include some text (e.g., as captions, labels, or navigational instructions), in some embodiments, products of the invention are substantially visual, which can be taken to mean that the products do not include or require expository paragraphs of text for understanding.
  • a visual curriculum has benefits due to the fact that many people learn in different styles and also that many scientific concepts are conducive to teaching visually. Additionally, a visual mini-curriculum is easier to distribute to audiences with different languages, since chapters of text do not need to be translated.
  • a mini-curriculum generally defines teaching material in that content is organized according to some pedagogical principle. For example, it may be determined that it is preferable to teach DNA replication prior to teaching mutation, and all prior to teaching population genetic concepts relating to diversity but after teaching Mendelian genetics. Accordingly, a visualization product may be prepared that includes, and indeed centers on, replication and reproduction as the molecular basis for inheritance, but the visualization product may follow a sequence that begins with Punnett square before giving the molecular mechanisms of diploid genetics. The sequence may end with illustrations linking the inherited alleles to populations in a geographical context.
  • a mini-curriculum may be prepared that presents a visualization of a molecular process such as apoptosis but the pedagogical organization may include assessment actions built in to the visualization and linked to certain parts of the illustrated apoptosis mechanism.
  • the assessment tool could be, for example, an on-screen test (e.g., click a multiple-choice answer to proceed).
  • the assessment tool is embedded as an interaction requiring a viewer to influence the depicted scene in the scientifically correct mechanism.
  • assessment includes visual aspects and a user's progress is assessed visually. A user may interact with a visual display to satisfy an assessment (e.g., drag and drop the appropriate molecule given context). In this way, visual assessment can capture the assessment of a user.
  • the visual assessment embodiments are included but not limited to: 1) allowing student to visually modify existing imagery (either through labeling, additional sketching, selection or other activities), 2) order sets of still images or image sequences (animations) to properly sequence a temporal process, 3) create their own custom imagery within the system, control parameters that impact the quantitative and/or qualitative output of simulations and game-like interactives.
  • Systems and methods of the invention not only allow instructors to monitor student progress and understanding within and across individual assets, but they also enable/guide them in implementing asset-based activities in a flipped-classroom context.
  • aspects of certain digital assets are designed to be used by students at home for instructional purposes, while other aspects of these assets are designed to facilitate classroom-based discussions and problem solving.
  • the invention offers a new level of transparency to users that is realized at two levels: a) the sources used for creation of content in all form (structural, dynamic or other) and b) the process and methodology used to create the visualization itself.
  • Systems and methods of the invention provide for rapid updating of content based on changing scientific data or shifting theories within the scientific community.
  • the system designed to allow revisions within digital assets as well as deletion or creation of entirely new digital assets.
  • assessments may be provided with or within visual products.
  • a visualization may be accompanied by a test that prompts a user to make a series of answers in an extrinsic medium.
  • a user could provide written answers outside of the system while accessing a visual product.
  • assessments are adaptive and embedded within a visual product. For example, in illustrating a molecular biology reaction, a user may have to drag the appropriate molecule into a scene, e.g., from a palette of candidate molecules.
  • the assessment can aid in evaluating a student by, for example, measuring progress through educational objectives.
  • the invention generally relates to systems and methods for visualizing an entire biological entity and particularly to an illustrative and comprehensive visual model of a cell or cell type, method for studying cells or associated model organism from which cells are isolated and studied.
  • the invention provides methods for representing biological entities using interconnected visualization digital assets to depict substantially all components and processes of the modeled entities and to tailor the depiction to an education level of an audience.
  • Individual components are modeled from scientifically accurate data and animation and simulation techniques are used to illustrate the actions and interactions of those components that make up the biological processes. Levels of detail and presentation styles may be automatically tailored to the education level.
  • the invention generally relates to systems and methods for providing a narrative visual depiction of substantially all components and processes related to an entire cell, such as a prokaryotic or eukaryotic cell.
  • the components and processes are not limited to being inside of a cell, and systems and methods of the invention also show extracellular components and events in which the cell interacts with extracellular entities, such as bacteria, viruses, antibodies, proteins, and other biological molecules.
  • the narrative visualizations in the aggregate will generally include substantially all of the components and processes associated with a single cell, different cell types, methods for studying cells and relevant model organisms from which cells are isolated and studied.
  • the Visual Cell is presented using a plurality of interconnected digital visualization digital assets.
  • the visualizations are made using curated models that visually convey parts of components or processes associated with the cell.
  • the visualization digital assets may be animations, one or more stills, interactive games, etc.
  • the visualization digital assets can be made using rigged models, in which a rigged model includes data representing a biological structure and a rig that defines animation dynamics for the biological structure.
  • the visualization digital assets can be made using simulations, in which the model's biological structure and conformations is derived through various methods of simulation.
  • the visualizations can be tailored to an education level of a user or a chosen level of complexity.
  • cellular phenomena are depicted and communicated in series of visual mini curricula.
  • the number and order of digital assets within these mini-curricula can be customized by instructors thereby improving on the rigid 'Table of Contents' structure characteristic of print textbooks.
  • the modularity of the system lends itself to a digital badging system whereby students, as a result of their performance on embedded visual assessments, will receive and display digital badge credentials within the Visual Cell to demonstrate mastery of biological concepts.
  • Embodiments of the invention may provide The Visual Cell, an online, immersive and interactive learning environment for the most challenging concepts in the life sciences (including but not limited to cell and molecular biology, biochemistry, developmental biology,
  • the system is organized into visual mini-curricula and topic- specific collections and built upon a digital library of models, customizable imagery, animations, interactives and assessments.
  • the system offers various learning paths through the material that tailor the materials to various educational levels including AP-Biology, introductory and advanced college biology topics.
  • Data-driven scientific visualization digital assets are also available to scientists, educators and publishers in the context of topic-specific collections.
  • a eukaryotic cell may include a nucleus, chromosomes, ribosomes, microtubules, microfilaments, centrioles, cilia, flagella, and other structures.
  • An animal-based cell may include organelles such as the nucleus, the nucleolus (within the nucleus), rough and smooth endoplasmic reticulum, Golgi apparatus, mitochondria, vesicles, lysosomes, centrosomes, centrioles and other such structures.
  • a plant cell may include a cellulose cell wall, a large central vacuole, and chloroplasts, as well as organelles.
  • Fungal cells may include chitinous cell walls.
  • a bacterial cell will typically include a cell wall (e.g., thick peptidoglycan for Gram + bacteria), a plasma membrane, such extracellular structures as fimbriae and pili, S-layers, glycocalyx, and flagella.
  • Intracellular bacterial components include the bacterial chromosome and plasmids ribosomes and other multi-protein complexes, intracellular membranes, cytoskeleton, as well as nutrient storage structures such as inclusions, vacuoles, or other micro-compartments.
  • Archaea cells may have a lipid monolayer membrane.
  • Processes in a cell that may be included in a visual cell include molecular transport, DNA replication, reproduction, protein synthesis, metabolism, and signaling. Particular processes that may be included are exocytosis, endocytosis, phagocytosis, mitosis, meiosis, transcription, translation, ATP synthesis, electron transport, dinucleotide redox, photosynthesis, a great variety of bacterial metabolic pathways (e.g., methanogenesis, aerobic respiration, etc.), chemical signaling, and receptor-mediated signaling, among others. Components and processes of cells may be found discussed in Cooper and Hausmann, 2013, The Cell: A Molecular Approach, 6th Ed., Sinauer Associates, Inc., 832 pages, and in Karp, 2013, Cell and Molecular Biology:
  • FIG. 26 illustrates an embodiment of the invention showing one of several options for users to interact with the Visual Cell as presented on the display of an electronic device 2601.
  • Device 2601 may be used by an audience member (e.g., student or other person) to interact with the cell.
  • Interaction can include, for example, zooming in to examine features of interest, triggering metabolic phenomena and simulations, or selection of graphical navigational hotspots that launch educational digital assets.
  • systems and methods of the invention can be used to provide models that cover a substantial entirety of a variety of complex natural systems, including, for example, organs, whole organisms, ecological concepts, and other systems.
  • methods and systems of the invention are used to provide a visual cell.
  • the Visual Cell uses interconnected visualization digital assets to represent components and processes of the cell.
  • the visualization digital assets may be animations, one or more stills, an interactive game, and interactive animation, etc.
  • Individual components, such as proteins or other macromolecules, for example, may be depicted using rigged or simulated models.
  • a rigged model discussed in greater detail below— includes geometry representing some depicted structure and a rig.
  • a rig is known in the art of 3D animation and generally refers to an organized system of deformers, expressions, and controls applied to an object so that it can be effectively animated.
  • Rig has been used in the animation arts to include a deformation engine that specifies how movement of a model should translate into animation of a depicted entity based on the model.
  • a rig provides software and data used to deform or transform a neutral pose of a model into a specific active pose variations. By having animation software manipulate a rig
  • a curated model which includes a rig based on joints/bones or particles, may be made using an animation environment such as Maya.
  • a rig may be saved as a file that includes a reference to the rigged geometry file.
  • Rigged models are used in the digital visualization digital assets that make up The Visual Cell.
  • the Visual Cell can be presented on an electronic display to allow a user to zoom in or out and to focus on areas of interest, to study particular phenomenon, and to see individual components of a cell with the context of the entire cell as a living organism.
  • FIG. 27 illustrates a zoomed-in level in which a user may set a level of detail, or concentration, to be included in the scene.
  • a slider is depicted at the bottom of the screen for populating the cellular environment with a desired level of detail.
  • cellular mega-structures such as the endoplasmic reticulum and endosymbiotic organelles are depicted.
  • FIG. 28 shows a cellular environment in which a user has increased a level of complexity and the cell is depicted with substantially all macromolecules present.
  • a menu is set off to the side to show, for example, how to select and highlight individual molecular families within the cellular environment using color-coding.
  • the menu can indicate that all nucleic acids are blue all proteins are red, all lipids are yellow, and all carbohydrates are green. Additionally, the menu can be used to toggle the appearance of certain classes of macromolecule on and off. Since each macromolecule is represented using a rigged model 121 built on data that is sourced from primary scientific research, the actions and interactions depicted within the cellular environment have a high degree of scientific accuracy.
  • Each rigged model can include information about the source of the supporting data for that model.
  • Use of The Visual Cell can include viewing the source of the data (e.g., hover a mouse over a protein and see a pop-up listing that protein's PDB accession number, a citation to a publication describing the structure, as well as visualization methods used to curate, animate, simulate the protein).
  • viewing the source of the data e.g., hover a mouse over a protein and see a pop-up listing that protein's PDB accession number, a citation to a publication describing the structure, as well as visualization methods used to curate, animate, simulate the protein.
  • visualizations can be shown with a consistent visual style.
  • the invention provides methods of interacting with The Visual Cell to satisfy different interests and objectives. Different objectives such as open curiosity versus seeking a specified learning objective may be best served by different interaction methods. Systems and methods of the invention provide for differing interaction approaches that include visual searching, "choose your own adventure” interactions, interactions guided by educational standards, or planned scientific modeling.
  • Searching provides a method of interacting with The Visual Cell as a reference tool.
  • a user may search for and retrieve one of the visualization digital assets based on a specific topic. For example, a user may perform a search for a biological concept (e.g., "Okazaki fragments"), thereby causing the system to display a subset of the plurality of interconnected visualization digital assets that relate to the biological concept (e.g., replication).
  • a biological concept e.g., "Okazaki fragments”
  • a "choose your own adventure” style interaction may be used to interact with certain visualization digital assets of The Visual Cell.
  • This interaction style can include choosing at least one option, to cause the system to select at least one visualization digital asset for display. The selection may proceed down a decision tree. Choosing the first option may cause the system to present a further choice, and so on, leading the system to display a final subset of the plurality of interconnected visualization digital assets.
  • This style of interaction may be suited to building a game based on The Visual Cell. It may be found that the choose-your-own-adventure style is also well suited to a tool for casual browsing by the interested lay public who are curious to explore the cell in an open way.
  • Educational objectives may be used to structure an interaction with The Visual Cell. For example, interacting with The Visual Cell may specifically be guided to track a curriculum or a standards-based educational plan. While the entire Visual Cell may be available in memory, certain visualization digital assets are selected for display by the system based on an educational standard.
  • An educational standard e.g., as published by a governing entity
  • Educational objective interaction further includes tailoring the presentation of The Visual Cell to an educational level or objective. For example, one user may navigate through some of the visualization digital assets at a first level of complexity corresponding to a first educational level of that user.
  • the Visual Cell product may have application in both the high school and the college classroom. Additionally or alternatively, The Visual Cell may adapt itself as a tool to a single user as that user progresses in their understanding of cellular phenomenon. Moreover, The Visual Cell may adapt its presentation (e.g., level of complexity) in response to the results of embedded assessments (discussed in greater detail below).
  • Modeling or testing an idea in silico is provided for via interacting with The Visual Cell.
  • the Visual Cell can be used to present two competing models of a biological phenomenon.
  • a user may change parameters of the cellular environment to observe how the cell responds (e.g., at what extracellular osmolality will this cell burst?).
  • Such parameters include, for example, temperature, pH, [02], pressure, salinity, viscosity, and others.
  • the Visual Cell tailors the materials that it presents to its users in several different ways.
  • Each digital asset within The Visual Cell database is 'rated' by curricular and complexity level. Therefore while a set of 20 digital assets might cover a specific topic like 'Biomembranes', only 5 of them would be rated/tagged for inclusion in an AP-Bio learning path/classroom, 12 would be tagged for use in Introductory Biology and the remaining 3 would be advanced digital assets (for advanced students or scientists).
  • subsets of widgets can also be grouped according to another cross-cutting themes such as evolution or topics that align themselves with educational standards. For example, an instructor may want to assemble a set of widgets that focuses on evolutionary aspects of biology and thereby find and assign widgets that are tagged with this underlying theme, but belong to many different specific mini-curriculum topics (i.e.
  • the Visual Cell i.e. a 'learning path' through the material of that mini-curriculum.
  • This order of widgets (as noted above it could be 5 or 12 or 20 depending on level of the audience), however, can also be customized by the instructor.
  • the UI of The Visual Cell allows instructors to inspect a library of all widgets within a mini-curriculum (represented with small icons and titles), individually drag-and-drop these widgets onto a custom learning path template and then assign this new custom order to their virtual classroom.
  • Customization in The Visual Cell can also be found at the level of individual digital assets and their functionalities. For example, digital assets that allow students to create their own custom 'recipes' for assembling a molecular landscape image are unique in that the output of these activities are completely custom to each student (no two images generated by these digital assets are likely to ever be the same since the student recipe drives a simulation to position molecular components into the 3D scene and resulting image).
  • the study portfolio is also used by the student to write notes and associate them with collected materials as preparation for class tests ('collected materials' in The Visual Cell might be a snapshot from an instructional movie, or a custom image or model created within an assessment digital asset or the result of a simulation launched by the student).
  • Instructors are able to review students' digital study portfolios and make edits (or suggest edits for the students to make).
  • the creation and review of these study portfolios is a valuable assessment activity in itself since it gives instructors a good sense of what the student understands about the material as well as any misconceptions they may have. Since each student's activities, associated notes and assessments is unique to that student, the study portfolio is another example of a tailored experience within The Visual Cell.
  • visual assessment may be included in The Visual Cell and its component digital assets.
  • Visual assessment paradigms within the digital assets of The Visual Cell leverage the richness of the varied forms of visual media that we use for instructional purposes. They offer a unique opportunity to reuse and cast in a different light visual activities driven by students that tie back to the instructional materials and therefore offer a more consistent experience to the student.
  • Example of such visual assessment digital assets may include any of the following.
  • the interactive labeling of a figure or diagram by a student (a visual similar or identical to those they have previously encountered in an instructional digital asset but that they are now challenged to label, annotate and/or comment on).
  • An interactive digital asset challenges students to re-order a jumbled sequence of visuals (either static frames/slides or movie segments) in order to assemble the proper movie of an ordered process.
  • This type of activity reinforces the concepts by revisiting the visual materials previously used in an instructional digital asset but also tests the student's understanding of the chronological order of a complex process (for example, given 6 movie clips across the gene-to- protein continuum, the student is asked to order them such that the complete movie proceeds from gene transcription, RNA maturation and splicing, nuclear export, translation and protein folding).
  • the movie is integrated within a personalized digital study portfolio— a continuously updated and instructor-moderated document that students use in preparation for larger exams throughout the course.
  • One possible digital asset provides free-hand annotations of existing visuals (e.g., 'circle the cytoplasmic domain of this protein').
  • 'guided sketching' activities the freedom of such assessment provides instructors with an even broader window on what students are thinking (compare, for example, the results of the 'Picturing to Learn' NSF project led by Felice Frankel).
  • a 'create-your-own-study-figure' interactive digital asset can be included that lets students create their own custom, professional-quality image or simple animation and save it as part of their online study portfolio.
  • a list of molecular ingredients is presented to student along with a challenge: 'model a red blood cell membrane' (these challenges are based on (i) the instructional materials previously presented and (ii) tailored to the curricular level.
  • Student has the ability to selectively include individual ingredients along with their relative amounts and submit this recipe to The Visual Cell via the web UI.
  • the student's recipe is used to assemble and, in some cases, simulate a custom 3D model in an automated fashion. This complex model is then automatically rendered into a beautiful image that is sent back to the student via the web UI.
  • Such custom scientific images and short animations are not only creative visual assessments where students have control over their creation but, in doing so, the system tests their
  • visual assessment harnesses the benefits of visual thinking in students. It broadens the scope of assessments because many more concepts (and misconceptions) can be gleaned from students through the use of a multitude of visuals, whether static, interactive or custom-created by students themselves.
  • the richness of visual assessment activities also has the potential to change the paradigm of 'learn, learn, learn, learn, learn, asses' model to a demonstratively more effective type of instruction (in terms of understanding and long-term retention) which follows the 'learn, asses, learn, asses, learn, asses' model. The latter begins to blur the distinction between learning and assessment phases as a result of the rich experience provided by creative visual assessments.
  • the visual assessment embodiments are included but not limited to: 1) allowing student to visually modify existing imagery (either through labeling, additional sketching, selection or other activities), 2) order sets of still images or image sequences (animations) to properly sequence a temporal process, 3) create their own custom imagery within the system, control parameters that impact the quantitative and/or qualitative output of simulations and game-like interactives.
  • Digital assets of the Visual Cell not only allow instructors to monitor student progress and understanding within and across individual assets, but they also enable/guide them in implementing asset-based activities in a flipped-classroom context. For example, aspects of certain digital assets are designed to be used by students at home for instructional purposes, while other aspects of these assets are designed to facilitate classroom-based discussions and problem solving.
  • the Visual Cell offers a new level of transparency to users that is realized at two levels: a) the sources used for creation of content in all form (structural, dynamic or other) and b) the process and methodology used to create the visualization itself.
  • the modularity of the Visual Cell platform enables rapid updating of content based on changing scientific data or shifting theories within the scientific community.
  • the system designed to allow revisions within digital assets as well as deletion or creation of entirely new digital assets.
  • FIG. 1 generally illustrates an architecture of systems of the invention that can be used to provide The Visual Cell.
  • the Visual Cell generally includes a plurality of digital visualization digital assets.
  • the Visual Cell, and the included plurality of digital visualization digital assets may be made by drawing on an asset database 105.
  • Digital assets within digital assets database 105 generally refer to an image, an animation, an interactive diagram, a mini-game, or such a piece of digital media.
  • a digital asset will include one or more curated models from a curated model database 109.
  • Curated database 109 generally includes one or a plurality of rigged curated models 121.
  • a curated model may generally be understood to refer to a multi-dimensional (e.g., 3D molecular) model that integrates scientific information (structural, dynamic, and other) that is 'ready to use' for visualization.
  • Curated models 121 may be built de novo or by sourcing scientific data from a suitable source such as, for example, a simulation, structural data (e.g., from protein data bank), dynamic data, or the scientific literature. Curation includes selection or building of a model and rigging the model to produce rigged model 121. Rigging a model, whether using a method such as bones and joints or particles, can make a model 'ready to use' for visualization.
  • FIG. 2 gives an exemplary structure of a rigged model 121.
  • a rigged model 121 will generally include a model 203 and a rig 207. Rigging with joints and bones is discussed in greater detail below (e.g., with respect to FIGS. 7-10).
  • Model 203 includes data representing a structure, often in the form of a geometry file. Any suitable model 203 may be included in a rigged model 121.
  • model 203 is a geometry file of a format suitable for creation, viewing, and manipulation within modeling or animation software such as, for example, Autodesk Maya. Any suitable animation software may be used.
  • Exemplary animation software products include those provided by Cinema4D Studio by Maxon Computer Inc. (Newbury Park, CA), Blender supported by the Stichting Blender Foundation (Amsterdam, the Netherlands), and 3DS Max 2014 by Autodesk, Inc. (San Rafael, CA).
  • geometry files can be imported from sources such as structure database, created de novo within a modeling environment, or built of raw data obtained from an experiment or assay.
  • the structures to be represented by geometry files may be predicted by computational algorithms, or may represent real structures determined by spectroscopic methods such as X-ray crystallography or nuclear magnetic resonance (NMR).
  • One exemplary approach to obtaining geometry files includes the use of a molecular graphics application such as Chimera or PyMOL.
  • Other suitable applications may include Astex Viewer, UGENE, DS Visualizer, Swiss PDB Viewer, Interchem, VMD, RasMol, Jmol, Python Molecular Viewer, Coot, MDL Chime, MolSoft Viewer, and other such products.
  • Such a program can be used to open raw structural data, such as a set of coordinates from a protein databank (PDB) file and to export the structural data in a format suitable for use in a modeling environment.
  • PDB protein databank
  • a PDB file embodies a format for representing actual 3D structures of biological molecules.
  • the PDB format is widely accepted as a standard in the biosciences.
  • the molecules may include protein or a nucleic acid (RNA or DNA), a complex of several proteins, a complex of protein with nucleic acid, or any of these in a complex with small molecule ligands such as drugs, cofactors, metal ions, etc.
  • the 3D structure of the macromolecule is usually determined by X-ray crystallography, but other spectroscopic methods, such as NMR, are occasionally employed.
  • the Protein Data Bank currently archives over 90,000 PDB files of macromolecular structures, which are freely available to the public. See, e.g., Berman, et al., 2000, The Protein Data Bank, Nucl Acids Res 28(l):235-242.
  • the PDB format includes ASCII text giving XYZ coordinates for atom locations, as well as data on atom-to-atom bond connections. Other information typically included are protein amino acid sequence and secondary structure, crystallographic space group, and general comments on the biological role of the protein. Molecular graphics applications such as Chimera or PyMOL by design readily import PDB files.
  • the structural data can be exported from the molecular graphics application (e.g., Chimera, PyMOL) to generate geometry files. These may be exported as Virtual Reality
  • Modeling Language VRML
  • OBJ format a common data format for 3D data
  • scripts can be used to prepare a geometry file from a set of coordinates using, for example, Maya Embedded Language (MEL) or the Python programming language.
  • MEL Maya Embedded Language
  • the method to use may relate to what will be done with the geometry once inside Maya.
  • large PDB datasets are brought into Maya as geometry files using the multi-scale model feature of Chimera.
  • structural data can be obtained for modeling using the product Molecular Maya Toolkit, sometimes referred to as m Maya or Molecular Maya.
  • Molecular Maya is a free software toolkit that extends the capabilities of Maya by allowing users to import, build, and animate molecular structures.
  • Molecular Maya includes the functionality to open PDB- formatted files.
  • Molecular Maya works with Maya 2011, 2012, 2013, and 2014 and adds a molecule- shaped icon to the Maya environment.
  • Molecular Maya includes (or adds to Maya) UI elements for opening PDB files.
  • Molecular Maya can import the text-formatted native PDB file.
  • Maya or Molecular Maya
  • PDB file Once imported, it can be viewed as atoms. However, Molecular Maya can transform it into a geometric structure, with options for selecting levels of resolution. Once imported, the geometry file provides the model 203 for a rigged model 121.
  • FIG. 3 diagrams a method 301 for curating a molecular model.
  • Method 301 operates preferably within the context of determining an objective 321 such as assembling a full-length protein molecular model from smaller or incomplete pieces of structural data.
  • a computer system is used to obtain 327 structure data.
  • Structure data can be obtained from a scientific assay such as x-ray crystallography, either directly or once published (e.g., from PDB files).
  • the structure is used to build 331 a model 203, typically a geometry file.
  • the model may optionally be skinned 335 with textures or shaders.
  • the geometry is rigged 339 with a rig that defines animation dynamics for the structure such that a range of motion for the rigged model is defined (i.e., for the depiction of the underlying structure in a downstream animation).
  • Each curated model is accessioned 343 to curated model database 109. Access to these rigged, digital models 121 is then provided for use in illustrating scientific concepts. Access may be provided through, for example, the
  • visualization digital asset database 105 in which one or more rigged model 121 may be bundled into visualization digital assets.
  • digital asset database 105 and curated model database 109 may be operated by interaction through a computer system to perform methods such as method 201. Any suitable computer system may be used.
  • FIG. 4 illustrates components of a computer system 401 that may be included in systems of the invention.
  • digital asset database 105 operates with the ability to connect to and pull from curated model database 109.
  • a curation computer device 407 is used to create curated models and populate model database 109.
  • Computer device 409 is used to build digital visualization digital assets that use the rigged models 121.
  • curation computer device 407 and computer device 409 are being described in terms of their roles. These roles can each separately be performed by using one or any number of different computers and can even both be performed through the use of a single computer.
  • a computer generally refers to a device that includes a processor coupled to a non-transitory memory and an input output device.
  • Computers of system 401 may communicate with one other via a network— broadly referring to the hardware used in transferring signals between computers.
  • Network 401 may be taken to include internet hardware such as telephone lines, cell towers, local switches and routers (e.g., LINKSYS products by Cisco Systems, Inc. (San Jose, CA), Ethernet cables, Wi-Fi cards, network interface cards, and other such device.
  • Network 401 may be understood as providing the ability to obtain structures from a structure database such as, for example, protein databank.
  • system 401 provides a construction computer device 423 for constructing a visual product (which device may be provided by one or more separate, dedicated devices or may be provided by the same one or more computer device providing either or both of curation computer device 407 and computer device 409).
  • curation computer device 407 and computer device 409 are employed in a production environment, wherein skilled scientist-animators rig models and build assets.
  • curation method 301 is performed using curation computer device 407.
  • the output of curation method 301 will include at least one rigged model 121.
  • construction computer device 423 refers to the personal computer (e.g., tablet, laptop, or desktop) used by a consumer to log into system 401 and initiate a request to interact with The Visual Cell.
  • the Visual Cell generally includes substantially every component of a single cell, and each may be modeled using a rigged model— i.e., with a structure and a rig.
  • FIG. 5 shows a modeling and animation tool as presented by system 401, i.e., a screenshot from a modeling and animation software product (e.g., as implemented on curation computer device 407).
  • the animation software is provided by Molecular Maya (e.g., m Maya v 1.0) in combination with Autodesk Maya and is used to rig a model such as may be obtained from a PDB file.
  • System 401 can be used to create models, rig models, and create visual products such as animations that use those models.
  • a model— or geometry file— can be rigged, and the rig will typically include a reference to the geometry file, it will be appreciate that a rig can be changed to reference a different model. That is, one of the valuable properties of a rig is that it can be used with one geometry then another. For example, a modeler could make a "quick and dirty" geometry and hand it off to the rigger. The rigger could build a rig using that geometry while the modeler works on a more detailed geometry. However, as used within an animation, a rig will generally reference one model (i.e., the geometry that it rigs).
  • system 401 includes Maya and models 203 are represented through the use of Maya's dependency graph.
  • Geometric objects, as well as data processing units such as transforms and shaders, are encapsulated as nodes. These nodes are connected through their attributes into a network that is known as the dependency graph.
  • Each node is dependent upon another, which includes that as the dependency graph is dynamically updated, changes to any node automatically propagate through the graph to all other nodes which are dependent on it.
  • This dynamic updating of the dependency graph is the core of the real-time graphics engine of Maya.
  • a Maya scene is a system of interconnected nodes that are packets of data. The data within a node tells Maya what exists in a scene.
  • Maya contains special node types (e.g., directed acyclic graph nodes) for certain things.
  • DAG nodes when working on objects in Maya's viewport, those objects, such as cubes, spheres, and planes of surface geometry, are DAG nodes.
  • a DAG node is model of two types of nodes, transform and shape nodes.
  • a shape nodes describes what an object is and a transform node describes where it is.
  • a model includes all of the structures and their locations needed to represent the intended object, and those structures can be, for example, nodes within a Maya dependency graph.
  • FIG. 6 shows a protein model 601 according to certain embodiments. If created in Maya or Molecular Maya, each polygon or non-uniform rotation b-spline (NURBS) curve of model 601 may be included as a node in the dependency graph.
  • NURBS non-uniform rotation b-spline
  • a complex geometry such as model 601 can be obtained by building within Maya or by import. For example, as discussed above, PDB files can be imported directly into Molecular Maya or by exporting VMRL from a molecular viewer. Additionally, complex geometries can be built within Maya using tools for 3D modeling.
  • a 3D model includes the geometry provided by surface. Maya supports three surface types: polygons, NURBS, and subdivisions.
  • a polygon geometry includes a surface made up of polygon faces with shared edges and vertices. Polygonal surfaces can be split, removed, extruded, and smoothed.
  • NURBS geometries which basically comprises surfaces created over a network of NURBS curves and converted to triangles when rendered.
  • Subdivision surfaces, or subDs are a way of adding detail to particular sections of a mesh by subdividing the existing surfaces.
  • Systems and methods of the invention relate to depicting substantially all of the components and processes associated with a single cell, various cell types, methods for studying cells and relevant model organisms from which cells are isolated and studied.
  • some materials are unambiguously within and of the cell—such as the genomic nucleic acid of that cell.
  • the comprehensive cellular model may also include extracellular signaling molecules, parasitic and endosymbiotic intracellular guests, viruses that affect the cell, transient materials such as chemicals that freely diffuse across the membrane, and even neighboring cells.
  • Individual components are represented using rigged models.
  • rigging a low resolution approximated model of a viral protein is used due to its simplicity and ease of visualization in the accompanying figures.
  • the disclosed rigging techniques may be applied to substantially all cellular components.
  • FIG. 7 depicts a model 701 representing a low resolution approximated model of a reovirus sigmal protein.
  • Reovirus attaches to cellular receptors with the sigmal protein, a fiberlike molecule protruding from the 12 vertices of the icosahedral virion.
  • the receptor-binding fragment of sigmal includes an elongated trimer with two domains: a compact head with a beta- barrel fold and a fibrous tail containing a triple beta-spiral. See Chappell, et al., 2002, Crystal structure of reovirus attachment protein sigmal reveals evolutionary relationship to adenovirus fiber, EMBO J 21: 1-11.
  • Model 701 can be made by any suitable method such as, for example, drawing a NURBS curve and rotating it around the Y axis.
  • model 701 is made by importing data from a PDB file, specifically from PDB # 1KKE.
  • a PDB file can be imported directly into a program such as molecular Maya or a PDB file can be opened in a viewer (e.g., PyMOL) and exported as VRML which can then be opened by a program such as Maya or molecular Maya to arrive at model 701 as shown in FIG. 7.
  • Model 701 represents one subunit of the sigmal trimer and the beta-barrel head and fibrous tail are visible. That structure is represented here as a plurality of NURBS curves 705 defining a surface 709. This model 701 provides the geometry file that can be rigged for animation.
  • FIG. 8 illustrates one method of rigging model 701 using joints and bones. Rigging includes the creation of organized systems of deformers, expressions, and controls applied to an object so that it can be animated well. A rig will allow an animator to create an animation without himself doing the rigging. That is, rigging is uncoupled from animation or simulation, allowing different tasks to be performed by specialists. As one of skill in the art will recognize, rigging is a continuously evolving practice.
  • rigging with joints and bones will include starting with a geometry such as model 701, building a skeleton, creating the rig and weighting the geometry. Rigging may also occur using a particle system which can subsequently be used for simulations that depict dynamic motion.
  • Joints are connected by bones 809, which are represented by wireframe pyramids with the point pointing towards the child when joints 805 are parented together.
  • a bone 809 will extend between a parent and a child joint 805.
  • a skeleton can be assembled to correspond substantially to a skeleton as known in zoology, however a skeleton more generally represents a structure for animation. In fact, a strength of the animation methods described herein is that the skeleton need not match the natural skeleton.
  • a skeleton may be bound to a skin so that, when bones and joints of a skeleton move (e.g., according to inputs and a rig), the skin presents a visible surface that deforms (e.g., according to how it is bound to the skeleton).
  • first joint 805a is created at the end of the fibrous tail of the monomer.
  • second joint 805b is created, bone 809a is created extending from first joint 805a to second joint 805b. This process is continued for all of model 701.
  • FIG. 9 shows model 701 with a set of joints 805 connected by bones 809 and a dialog box 901 for binding model 701.
  • FIG. 9 illustrates skinning the geometry of model 701.
  • Skinning geometry is the process in which geometry is bound to joints so that, as the joints are rotated or translated, the geometry is deformed.
  • the terms skinning and binding are generally interchangeable. Any type of binding by may be used such as, for example, smooth binding, interactive skin binding, and rigid binding.
  • each vertex of the geometry receives a weighted influence from the joints 805.
  • Interactive weighting allows the rigger to set weights by entering them.
  • the skeleton is bound to the geometry with the skeleton in the bind pose.
  • Controls can be created from locators or curves or any other node that can be selected in the viewport.
  • Other types of deformers may be used besides joint deformers and may include influence objects, lattice deformers, Maya Muscle, and other tools. Using bones and joints created during rigging, parts of a model can be moved with scientific accuracy.
  • FIG. 10 shows a motion of a model 701 based on the applied rigging.
  • the sigmal monomer has bent around joint 805.
  • Rotation around joints can be controlled by kinematic concepts, as provided for within animation environments such as Maya and molecular Maya.
  • Such animation environments provide for controls such as forward kinematic and inverse kinematic controls of systems of joints.
  • Forward kinematics refers to having each joint in a chain inherit the motion of its parent joint
  • inverse kinematics IK refers to causing joints to orient themselves based on the location of a goal known as an end effector.
  • an amino acid side chain in the active site of an enzyme may be rigged with inverse kinematics using the substrate as the end effector.
  • a protein subunit that undergoes a tertiary structure reorganization while changing conformations may be modeled using forward kinematics.
  • animation involves the use of deformers such as blend shapes.
  • a blend shape deformer allows a depicted structure to morph between two meshes and allows a user to control the blend and the morph.
  • at least two topologically identical meshes are created, representing the structure in at least two corresponding conformations.
  • a blend shape is created from the meshes and a node network is created that will work with constraints and rig controls to adjust the animated transformation between the two conformations.
  • the two meshes are selected and the Blendshape command is run from the Create Deformers menu.
  • a new node is created and one of the meshes can be deleted (now being represented by the Blendshape).
  • a rigged model includes an animation rig that is easy to understand.
  • controls are labeled and easy to select.
  • entering 0 in the translation channels for the controls return the rig to the start position.
  • IK handles use world space coordinates so setting translation channels to 0 moves the handle to origin.
  • the invention provide techniques that are suited for complex morphs that allow conformational states of proteins to be depicted.
  • animations that are based on actual data for protein dynamics to provide vibrations and degrees of flexibility that reflect the protein's actual range of thermodynamically- permissible motion.
  • the actual structural data is fed into the geometry of the 3D model 203, and dynamic data informs the rig 207.
  • rigged models provide a scientifically accurate range of motion for proteins and other structures, other benefits can be included such as collision detection or overlap prevention.
  • systems of the invention may be operable to register and warn against impending self-intersections through the use of self-aware rigging techniques applicable to scientific structures such as biological macromolecules.
  • collision detection rigging can include the use of electrostatic forces (e.g., as mapped to the surface of a space-filling model).
  • electrostatic forces e.g., as mapped to the surface of a space-filling model.
  • collision-detection rigging i.e., abiding by electrostatic concepts providing that like-charged surfaces repel and unlike-charges attract
  • the one or a set of MEL or Python scripts not only create Maya- native geometry directly from the PDB but also automatically create a rig that has some inherent motion constraints applied.
  • the automatic rigging may be applied with different types of molecular representation (ball & stick versus cartoon for example would have very different 'rules' applied to constrain motion).
  • a MEL and/or Python script can apply certain rigging to certain structural motifs automatically and by default. For example, the peptide bonds of a polypeptide can be automatically rigged for realistic rotations. The rigged model can be provided for "fine tuning" by a user by hand.
  • information for the rig is obtained from a scientific data source.
  • the conformational dynamics data bank (CDDB) can be accessed to obtain information about possible conformations of a protein.
  • a rig can be created to restrict the range of motion of the protein model to conformations allowed by the conformation data bank information.
  • a MEL and/or Python script can be used to automatically create that rig and apply it to the model based on CDDB data.
  • the CDDB is described in Kim, et al, 2011, Nucl Ac Res 29:D451-5. Suitable databases for protein dynamics may be discussed in Liu & Karimi, 2007, High-throughput modeling and analysis of protein structural dynamics, Brief Bioinform
  • curated models of the invention are suited for employment in modern gaming engines.
  • the digital assets (models, textures, rigs) used to develop high- end games are created in packages like Maya.
  • molecular-movie style animations are generated within an environment such as Maya for application within interactive molecular environments for educational purposes.
  • embodiments of the invention can use rigging concepts to depict motion through animation and can even be used to control levels of granularity at which motion can be depicted. For example, at one level, the overall motion of molecular structures within a cell can be shown, while at another level, motions at the atomic level can be depicted.
  • FIG. 11 illustrates protein dynamics at four different levels that can be illustrated using modeling, rigging and simulation concepts discussed herein.
  • the diffusion or random motion of entire proteins can be illustrated.
  • conformational changes associated with domains of proteins can be depicted, for example, within an animation provided by methods of the invention involving the use of rigged models.
  • the various side chain rotations of individual amino acids can be depicted.
  • the thermal vibrations of individual atoms can be depicted.
  • process such as diffusion and random motion or Brownian motion can be modeled as stochastic process and such processes can be implemented using computer programming or scripting.
  • MEL scripting or Python programming may be employed.
  • MEL or Python scripts start directly from a PDB coordinate file and generate ribbon or surface representations.
  • the MEL or Python scripts read from the PDB file, e.g., atom-by-atom.
  • a set of coordinates will be given to each atom and any bonds indicated in the PDB file will be treated as indicating a connection to another atom.
  • Shading groups are created in the Maya dependency graph.
  • MEL or Python scripts set shading for each atom and create a sphere in the dependency graph. For each bond, a cylinder is created.
  • These models created by MEL or Python scripts may be lighter and cleaner that exports from Chimera or PyMOL since they have been built within Maya using optimized types of geometry, such as NURBS, for example.
  • the geometry file once loaded into Maya appears as a structure in a display. For example, where a PDB file is imported, the protein molecule will be displayed (see FIG. 6 for an example).
  • the molecule in the display can be rotated, translated, and scaled using Maya's native functionality (e.g., hold down ALT + L, M, or R mouse button, respectively, while dragging) for transforming the scene view.
  • a molecule may be displayed using a known format such as a ball and stick model. Sticks represent bonds and balls represent atoms.
  • a molecule may be displayed using a surface model— i.e., showing a surface of the molecule.
  • methods of the invention are implemented by programming within an animation environment.
  • Maya provides an application programming interface, the Maya API.
  • MEL and the Maya API support construction of complex geometric objects, creation of new tools and workflows, and manipulation of object and tool attributes. Those programming mechanisms may be found discussed in Gould, 2003, Complete Maya
  • the API is used for large data sets and complex algorithms. Code accessing the API will be contained within a plug-in. Programming within Maya can be used to automatically import structures such as PDB files as geometries or to automatically rig geometries, as discussed above.
  • FIG. 12 diagrams a method 1201 of making a visualization digital asset for use in The Visual Cell that includes a plurality of curated models that can be used for conveying a scientific concept.
  • the depiction of a certain cellular concept is determined 1205 to be an important objective. Any suitable cellular concept may be depicted using methods of the invention including, for example, a cellular component or process from a plant, animal, bacteria, archaea, yeast, or any other.
  • a storyboard is developed 1209 that will determine the curated models to be produced or included.
  • curated models 121 are obtained 1213 from database 109 for inclusion in a visualization digital asset that lives in The Visual Cell.
  • the curated model(s) 121 to be included will relate to the cellular component or process to be represented.
  • an Archaea cell will require a lipid monolayer cell membrane and the lipids will have to be modeled appropriately.
  • Curated models 121 may be found to be particularly valuable for illustrated concepts that some students struggle with.
  • a visualization digital asset can be a pathway animation depicting a cascade of events in which at least two depicted biological structures interact only indirectly.
  • Depicting substantially all of the components of a single cell includes creating digital assets to depict a variety of cellular components and processes. For example, each cellular membrane and compartment, as well as all cellular proteins and nucleic acids, must be depicted. The metabolic and synthetic pathways must be represented as well as the signaling and information pathways. Each pathway and component will be represented by the models and animation choices best suited to those parts.
  • MAP kinase cascade may be well illustrated using a digital asset that includes an animation. Due to the nature of character rigging, indirect interactions can be understood.
  • MAP kinase kinases aka MAP2 kinases
  • upstream kinases e.g., MAP3 kinases
  • MAP3Ks such as c-Raf, MEKK4 or MLK3 themselves require multiple steps for activation.
  • MAP kinases exist that phosphorylate serine or threonine residues near proline on cytosolic proteins and also phosphorylate transcription factors during transcription.
  • an animation can illustrate the indirect interactions between, for example, c-Raf and a classical MAP kinase such as ERK1. Since each protein (c-Raf, a MAP2K, ERK1) is included with a structurally accurate model 203 and a dynamically accurate rig 207, an audience can view the indirect influence of c-Raf on transcription via an animation that is scientifically accurate. Additionally, this material can be illustrated through, for example, a web-based interactive decision tree, allowing a Georgia student to select input and decide conditions that control a depicted outcome. As discussed, a digital asset can include an animation.
  • Stills can be composed using models from database 109.
  • databases 109 For example, if a publisher wishes to illustrate the so-called central dogma of molecular biology within its cellular context to a high-school audience, systems and methods of the invention can be used to produce three stills, one to illustrate each of replication, transcription, and translation at their appropriate locations within the cell.
  • the nucleic acids and proteins can be included based on curated models from the database and the images can be stylized to communicate effectively with the high-school education level (e.g., bases can be presented in a simplified structure and each clearly labeled with one of A,T, C, and G).
  • a working researcher may desire a digital visualization digital asset consisting of a still image illustrating an autocatalytic property of a ribonucleic acid for publication in a peer- reviewed journal.
  • a still can be composed and— in view of the average post-doctoral education level of the readership— a valence electron cloud for the oxygen of a 2' hydroxyl group that acts as a nucleophile in phosphodiester cleavage can be illustrated and shaded so that readers visualize the ribozyme reaction mechanism.
  • systems and methods of the invention can be used to produce The Visual Cell that uses a number of digital visualization digital assets.
  • the Visual Cell, and those digital visualization digital assets may be tailored to an education level of an audience to effectively convey the phenomena associated with that cellular component or process.
  • FIG. 18 shows a DNA/chromatin strand in a stylistic manner in which progress through a modeling process is illustrated from top to bottom along the strand. At the top of FIG. 18, the DNA strand is represented by a simple curved line with a histone appearing a simple cylindrical spool shape. Moving down FIG.
  • the modeling of the DNA strand and the histone using polygons or NURBS curves is represented.
  • a user of certain interactive digital assets of the Visual Cell can build up such a model in, for example, without any knowledge of 3D animation software such as Maya or Molecular Maya. Detail is added and, at the bottom of FIG. 18, surface textures and shading and lighting is included in the model so that the packaging of DNA into chromatin is illustrated with a very high level of detail. The level of detail can be selected based on the education level of the audience.
  • FIG. 19 illustrates use of a product such as Molecular Maya to prepare model 203 showing DNA coiled around histones. Even though the final visual product may be determined to be in a digital asset that will be used by an elementary school audience, the highest possible level of scientific accuracy can be ensured by modeling the actors as rigged 3D structures.
  • Chromatin provides a good example due to the fact that the precise way in which DNA wraps around a histone is a product of the structure and dynamic properties of both DNA and the histone. Rigging techniques can be used to restrict the possible range of motion of the DNA realistically and then to wrap the DNA around the histone, as shown in FIG. 19. That model can then be used to create a visual digital asset that includes a still image.
  • a visual digital asset including this material can also include multiple layers so that the primary actors (here, DNA and histones) are depicted with scientific accuracy in their natural environment.
  • Using different layers can aid in automatically tailoring a visual product to the educational level of an audience. For example, where it is desired to teach simply the wrapping of DNA around histones, the back layer 2005 and mid layer 2009 can be put into soft focus so that the student's attention is given to the front layer 2013. Alternatively, a level of detail in mid layer 2009 can be increased for, for example, a journal publication about binding factors where the audience will typically have a post-doctoral education level.
  • a visual digital asset can be made—such as an animation, interaction, simulation, game, a photo-quality still, or similar material— that can be used to illustrate a scientific concept.
  • FIG. 21 presents an image that could custom-created by a user within an interactive digital asset of the Visual Cell using layered structure 2001, i.e., the final product of the methods illustrated by FIGS. 19-21. Since layers are used and since material is represented using rigged models, the image in FIG. 21 aids a user in understanding chromatin within the cellular context. The material is depicted with scientific accuracy and is tailored to the education level of an audience.
  • the Visual Cell includes tools for modeling multi-component systems within the cell with efficiency. Tools are useful where components are known to include numerous instance of like or very similar participles. Modeling tools may employ object oriented programming techniques to create, for example, all of the lipids within a lipid membrane. A phospholipid class may be created, and then for each lipid molecule, the class can be instantiated, and the instance can inherit the structure and rigging of the abstract superclass. Depending on the distance to the virtual camera (depicted here using color-coding) the phospholipid instances can be meshed with various levels of geometry detail thereby resulting a lighter, more memory-efficient 3D model.
  • FIG. 23-FIG. 25 illustrate membrane modeling according to embodiments of the invention to create a visualization product.
  • Concepts of the invention include the ability to set or control a level of detail in a visual product. Setting the level of detail can aid in creating a visual product with the highest possible level of scientific accuracy given the available inputs and can also help tailor a visual product to an education level of an audience.
  • a bilayer membrane is modeled.
  • a use can establish areas with different levels of detail over an abstracted grid for the membrane.
  • FIG. 23 shows use of Molecular Maya to establish different regions across a membrane and set progressively varying properties across those regions.
  • a different level of detail is being set (e.g., high level of detail may be set for a region that will be close to a camera).
  • each instance can be granted a level of detail based on the region within the membrane where it will be placed (i.e., so that areas of the membrane that are "close to the camera" have a higher level of detail and show some imperfect molecules or show molecules with a greater variety of shapes and motion).
  • FIG. 25 shows use of an electronic device 1601 to view a cellular membrane.
  • Systems and methods of the invention can be used to render an animation depicting a cellular
  • the 3D model and rigging of a rigged model 121 and any other inputs may be rendered into a bit- mapped based video clip.
  • the rendered animation may be viewed on a screen of 1601.
  • Visual Cell and its visualization digital assets can be tailored to an education level of an audience.
  • a level of complexity of the digital asset can be set according to an education level of an audience that will view an animation.
  • parts of the digital asset can be concealed based on the education level.
  • systems and methods of the invention are operable to
  • Tailoring to an education level can include controlling a number of elements to depict in an animation.
  • an animation depicting transcription initiation if the audience level is set at grade school, systems of the invention may depict only an RNA polymerase processing a DNA strand.
  • the system may include, for example, TATA binding proteins and transcription factors binding and recruiting the polymerase.
  • curated models or visualization digital assets may include elements or portions that are tagged with an education level so that systems may selectively exclude those elements or portions for education levels that do not match the tag.
  • an education level For example, in the biochemistry of cellular metabolism, it is thought that in an enzyme-catalyzed reaction, the substrate will fleetingly occupy a highest-energy transition state and that the nature of this transition state precludes its ever being observed according to quantum principles.
  • a model of the substrate may include rigging allowing the substrate to assume the transition state form and may further include rigging that vibrates or blurs the surface geometry at the instant the transition state form is assumed to prevent direct and instantaneous visualization of the transition state form.
  • the Visual Cell can be manipulated to represent different conditions. For example, global cytoplasmic parameters such as temperature, viscosity, salinity, or pH can be set (e.g., some proteins may exhibit different conformations, or some reactions may occur at different speeds, as such parameters vary). To expand, a number of proteins are known to respond to [H+] gradients. If, for example, an ATPase is being modeled in a lipid bi-layer membrane, a user may input a hydrogen ion concentration on either side of the membrane. If the concentration is isomolar the membrane, the ATPase— by virtue of its rigging— will be depicted as static. If there is a hydrogen ion concentration, the ATPase will be depicted as active.
  • global cytoplasmic parameters such as temperature, viscosity, salinity, or pH can be set (e.g., some proteins may exhibit different conformations, or some reactions may occur at different speeds, as such parameters vary).
  • a number of proteins are known to
  • system 401 includes a processor coupled to a non-transitory memory having stored therein a plurality of models, each model comprising data representing a structure and a rig that defines animation dynamics for the structure such that a range of motion of each model on an electronic display device 129 is predetermined without manipulation from a user.
  • FIG. 13 diagrams a method 1301 for constructing and providing a visualization digital asset that will live within the Visual Cell digital assets database 105.
  • method 1301 for providing a visualization digital asset includes determining 1305 some science concept or topic to be depicted.
  • a storyboard for the visualization may be developed 1309.
  • Curated models from the curated models database 109 are selected 1313 for inclusion based on the particular phenomenon being depicted.
  • At least one of the curated models will be capable of visually conveying at least a portion of the scientific concept.
  • At least one of the visualization digital assets will include a rigged model 121. Use of a rigged model 121 allows models to be animated 1317.
  • a visualization digital asset is constructed 1321 such that it includes at least one digital asset.
  • This visualization digital asset is then provided 1325 for use (e.g., for viewing by the audience on an electronic display device).
  • Method 1301 may include receiving data related to the education level of the audience.
  • the visual digital asset may be any digital asset that visually communicates a scientific concept relating to The Visual Cell.
  • the visual digital asset may be an animation depicted on a computer screen or it may include a tangible medium having files stored therein that can be accessed to view an animation.
  • the visual digital asset is a model of a cellular environment and a software interface that can be loaded onto a user computer device to allow the user to browse in, and interact with, the cellular environment.
  • the visual digital asset may live inside a digital textbook (e.g., for viewing via a tablet computer or similar device) that guides a user through, for example, a course in cell biology.
  • Providing the visual digital asset may include rendering an animation (e.g., taking the 3D modeling and animation files and outputting a video clip that comprises a series of bitmapped images).
  • the Visual Cell is tailored to an education level of an audience.
  • a user may initiate interaction with The Visual Cell (e.g., using a web interface) and may provide information such as grade level or age.
  • the Visual Cell can then be tailored to the grade level. For example, in some embodiments, tailoring The Visual Cell is done by automatically visually concealing one or more portions of a molecular model or cell environment based on the data related to the education level of the audience.
  • Constructing and providing The Visual Cell is preferably performed using a system that includes a processor coupled to a non-transitory memory.
  • the system can be used to construct an electronically displayable visualization product that comprises at least one digital asset that visually conveys at least a portion of a scientific concept.
  • the digital asset includes a curated model based on a structure 203 and a rig 207 and is tailored based on an education level of an audience.
  • An end user can access certain interactive digital assets of the Visual Cell system to initiate the custom creation of a visualization.
  • FIG. 14 depicts an interface 129 provided by computing device 125 for using systems of the invention.
  • systems and methods of the invention can employ a web front-end or other interface, such as a dedicated application, to allow users to access products described herein.
  • Embodiments of the invention to provide an easy-to-use interface for users to put in suggestions or requests. For example, a user can request their favorite cell type for The Visual Cell.
  • a user may see a web interface to set up a request for a visualization cell that emphasizes certain phenomenon for certain pedagogical objectives. While discussed above and throughout in terms of a single cell, any suitable scientific system may be illustrated by systems and methods of the invention.
  • embryonic development can be illustrated and conveyed by modeling a developing embryo using one or more rigged model 121.
  • Other systems suited for illustration by methods of the invention include organs, organ systems, populations, gross anatomy, viruses, and other concepts. Using methods of the invention as described herein, any of these scientific concepts and more can be illustrated.
  • FIG. 15 gives an overview of the web-based decision tree that exists within an advanced visualization digital asset of the Visual Cell which allows users to create custom visualizations using methodology 1501.
  • Method 1501 includes determining 1505 what product to make such as, for example, a still, a sequence, or an animation.
  • the environment is then constructed 1507. Constructing the environment includes layering 1509 (see FIG. 20) and selecting 1513 layer components. Layer component choices depend on the subject matter, the environment, and the layer. If a cellular biology concept is being communicated, options for components to have within various layers of the visual product may include none, nucleus, plasma membrane exterior, plasma membrane interior, mitochondrion, cytoplasm, others, or a combination thereof.
  • Embodiments of the invention include preset curated models and user-driven sets of layer components. The components can be customized and positioned 1517. Options are component- specific such as, for example, animation presets.
  • Rendering presets and color palettes are selected 1525.
  • Selecting color palettes can include assigning color by component or using an overall Kuler palette, and can also include using an overall image style (ambient occlusion (AO), simulated electron microscope (EM), cartoon- style, combinations).
  • Ambient occlusion is a method to approximate light shining onto a surface. Typically, ambient occlusion is used for realism.
  • Ambient occlusion models rays cast in every direction from a surface. Rays which reach the background increase the brightness of the surface, whereas a ray that hits an object contributes no illumination. As a result, points surrounded by a large amount of geometry are rendered dark, whereas points with little geometry on the visible hemisphere appear light.
  • Programs such as molecular Maya include shaders such as the EM shader to simulate the appearance of electron microscopy.
  • the product is watermarked 1533.
  • a delivery format is established.
  • the product may then be rendered 1537.
  • the resulting professional-quality custom visualization can then be embedded by the user into a study portfolio stored within the Visual Cell for further inspection and review by an instructor.
  • systems and methods of the invention can be used to create a variety of digital systems (The Visual Cell, organ, organism, solar system, machine, others).
  • Systems and methods of the invention may include additional features and functionality.
  • rigged animations can be used to depict and illustrate such diverse phenomena as polymerization, cell signaling, Brownian motion, lipid bilayer membrane structure, cellular organization, protein folding and conformation, organismal anatomy, embryonic development, bench-top lab experiment protocols, intracellular bio-molecular structure and composition, viral structure and function including capsid packing, the biochemistry of metabolism, phylogenetics, ecological principles, neural function, and other phenomenon.
  • models may be provided that illustrate
  • the audience may be any single person or group of people with any education level, and the invention addresses unmet needs for a variety of different audience types or education levels.
  • the audience may be of a collegiate or post-collegiate level, which may include for example, graduate, medical, post-doctoral or any other level.
  • Content may be provided that is relevant to pre-collegiate, undergraduate, graduate, medical school and post-doctoral.
  • high- school students e.g., in AP Biology
  • the Visual Cell provides support for the teachers as well as the students.
  • the curriculum component of The Visual Cell may include, for example, an assessment integrated into The Visual Cell.
  • systems and methods of the invention provide for collaborative learning.
  • content may be tailored to support paired, or groups of, students on projects.
  • Material may be delivered such that tasks or response prompts are directed to members of a pair or group to support collaborative learning objectives.
  • Curricula offered within The Visual Cell make the invention particularly valuable in an education context. For example, pre-med students can learn the effects of drugs on cell systems. For working research scientists, there is a need for the ability to provide scientifically accurate visualizations in which static or animated visuals are derived from actual datasets. scientistss may require a clear provenance of datasets used for a visualization.
  • the Visual Cell may be used by scientists to illustrate and understand competing models for mechanisms.
  • the general public may be well-served by books, articles, TV shows and documentaries that include a scientifically accurate visual cell tailored to the average education level of the general public within a market segment. For example, members of the public may derive great personal enjoyment and satisfaction from downloading and browsing a Visual Cell app for their computer device.
  • a society may be made safer and more mature if the public at large has tools for understanding the cellular phenomenon that make up their world.
  • a curriculum will generally include educational materials and preferably includes tools for assessment.
  • curricula being included with The Visual Cell is that, due to the visual nature of the cell, the curriculum need not be interwoven with prose exposition as required by convention for existing textbooks and journal articles. While the visualization digital assets of The Visual Cell may include some text (e.g., as captions, labels, or navigational instructions), it is preferably substantially visual, which can be taken to mean that The Visual Cell may not include or require expository paragraphs of text for understanding. Including curricula has benefits due to the fact that many people learn in different styles and also that many scientific concepts are conducive to teaching visually. Additionally, a visual mini-curriculum is easier to distribute to audiences with different languages, since chapters of text do not need to be translated.
  • visualization digital assets may include adaptive assessments embedded within.
  • a user may have to drag the appropriate molecule into a scene, e.g., from a palette of candidate molecules.
  • the assessment can aid in evaluating a student by, for example, measuring progress through educational objectives.
  • FIG. 16 depicts a storyboard 1601.
  • the table at the bottom of FIG. 16 gives an example of the varied and numerous structural pieces that are gathered and used to construct a curated model. That is, the table in FIG. 16 illustrates components that may be used in creating or populating a curated model database, which database may be used in the creation of storyboard 1601.
  • a teacher may wish to communicate death receptor mediated apoptosis to a college student audience.
  • the teacher may wish to communicate mechanisms of cell death through the binding of ligands to death receptors such as Fas/CD95 and the subsequent formation of the death-inducing signaling complex. See, e.g., Pennarum, et al., 2010, Playing the DISC: turning on the TRAIL death receptor-mediated apoptosis in cancer, Biochim Biophys
  • the teacher or a service provider may develop a storyboard to illustrate the Fas ligand member of the tumor necrosis factor (TNF) family and the aggregation of death domains (DDs), allowing Fas-associated death domain (FADD) to bind to the death domain of Fas.
  • FADD also binds to caspace-8 through its death effector domain (DED) and ultimately active caspase-8 is released to the cytosol.
  • Storyboard 1601 may include identified actors to be included in a visualization product.
  • a Table of Actors lists the Fas ligand (FasL); a tumor necrosis factor receptor superfamily member 10b (DR5); Fas; FADD; caspase-8; and caspase-5, as well as the primary structure and PDB accession number of each.
  • Storyboard 1601 is thus a valuable tool for planning a visualization and tailoring the visualization to the education level of an audience.
  • the teacher plans on illustrating apoptosis to a college-level audience.
  • the teacher or designer includes that information and provides the PDB numbers of the actors (proteins). This information allows a visualization product to be made and tailored.
  • FIG. 17 illustrates a curated model for DISC in the process of being built within the Maya/Molecular Maya UI.
  • the models are rigged so that the individual proteins move and assemble realistically.
  • Systems and methods of the invention may be used to create visual products that include still images.
  • FIG. 18 gives an example of how an underlying structural dataset can be used to drive multiple types of molecular representations— as seen in different top-to-bottom levels of the image.
  • the tailoring of the representation/rendering is something that would be specified and happen at the level of the specific digital module in the digital assets/modules database or Visual Cell.
  • FIG. 18 shows a DNA strand in a stylistic manner in which progress through the modeling process is illustrated from top to bottom along the strand.
  • the DNA strand is represented by a simple curved line with a histone appearing a simple cylindrical spool shape.
  • the modeling of the DNA strand and the histone using polygons or NURBS curves is represented.
  • FIG. 18 illustrates use of a product such as molecular Maya to prepare model 203 showing DNA coiled around histones. Even though the final visual product may be determined to be a single still image, the highest possible level of scientific accuracy can be ensured by modeling the actors as rigged 3D structures.
  • Chromatin provides a good example due to the fact that the precise way in which DNA wraps around a histone is a product of the structure and dynamic properties of both DNA and the histone. Rigging techniques can be used to restrict the possible range of motion of the DNA realistically and then to wrap the DNA around the histone, as shown in FIG. 19. That model can then be used to create a visual product that includes a still image.
  • a visual product including this material can also include multiple layers so that the primary actors (here, DNA and histones) are depicted with scientific accuracy in their natural environment. This functionality is provided via digital assets which may rest on a foundation of curated digital models.
  • FIG. 20 shows the layers of layered structure 2001.
  • a layered structure 2001 for conveying a scientific concept will include at least a back layer 2005, a mid-layer 2009, and a front layer 2013. Any suitable scientific concept can be depicted using layered structure 2001.
  • transcription initiation can be illustrated by having back layer 2005 provide a nuclear backdrop.
  • Mid layer 2009 can include environmental proteins as actors (e.g., one or more miscellaneous CCCTC -binding factor).
  • Front layer 2013 will generally include primary actors such as the "hero" protein, here, a transcription factor bound to DNA. Using different layers can aid in automatically tailoring a visual product to the educational level of an audience.
  • a curated database includes macroscopic models.
  • macroscopic curated models may be useful in the setting of 1) laboratory equipment and 2) biological organisms.
  • equipment could have curated models of lab equipment that are pre-rigged to animate in certain ways.
  • biological organisms most likely the model systems of biology... mouse, yeast, Xenopus, C. elegans, zebrafish etc.
  • models may also be pre- rigged to be 'animation-ready' .
  • rigged models of materials such as pipettors, streak plates, or dissection samples are used in a video-game engine style environment to allow a student to simulate an exercise one or a few times before performing the exercise with real equipment to save costs to an institution.
  • FIG. 23 shows use of molecular Maya to establish different regions across a membrane and set progressively varying properties across those regions.
  • a different level of detail is being set (e.g., high level of detail may be set for a region that will be close to a camera)
  • metadata i.e. an embedded property of a curated model within the database.
  • FIG. 24 illustrates bringing individual molecule models (themselves curated models) in to the membrane model.
  • Molecular Maya can then be used to populate the membrane with those molecules.
  • each phospholipid can be created as an instance of a phospholipid structure file that is rigged to allow appropriate rotation around bonds in the lipid tail.
  • the phospholipids can be "drawn" as a set of strokes using, for example, a MEL or Python script.
  • Transmembrane proteins and the membrane can be rigged to allow the proteins to float in the membrane and even displace laterally, if desired.
  • FIG. 25 shows use of an electronic device 1601 to view a visualization product of the curated models described in the invention.
  • the visualization product may include a rendered animation. That is, the 3D model and rigging of a rigged model 121 and any other inputs may be rendered into a bit-mapped based video clip.
  • the rendered animation may be viewed on a screen of 1601. Example 5.
  • Embodiments of the invention may provide The Visual Cell, an online, immersive and interactive learning environment for the most challenging concepts in the life sciences (including but not limited to cell & molecular biology, biochemistry, developmental biology, immunology, virology, neurobiology, physiology, experimental techniques and associated model organisms) - ones that are most effectively conveyed through visualization.
  • the system is organized into visual mini-curricula and topic-specific collections and built upon a digital library of models, customizable imagery, animations, interactives and assessments.
  • the system offers various learning paths through the material that tailor the materials to various educational levels including AP-Biology, introductory and advanced college biology topics.
  • Data-driven scientific visualization modules are also available to scientists, educators and publishers in the context of topic-specific collections.
  • FIG. 26 illustrates use of an electronic device to interact with a model of a complex natural system such as a whole cell model.
  • a whole cell model could be assembled from curated molecular models in the database.
  • Systems and methods of the invention can be used to provide models that cover a substantial entirety of a complex natural system.
  • an ecological system such as the water cycle or population biology can be modeled.
  • Astronomical and cosmological phenomenon such as energy and gravitational dynamics of galaxies and the space between them may be modeled.
  • Organisms may be modeled, such as substantially all of the organ system in a body, or the development of an organism over time (e.g., embryonic stages).
  • Neural networks and the architecture and function of the brain may be modeled.
  • methods and systems of the invention are used to provide a visual cell.
  • a rig representing some depicted structure and a rig.
  • these components are built and used within an animation environment such as Maya.
  • a rig may be saved as a file that includes a reference to the rigged geometry file.
  • FIG. 27 illustrates a zoomed-in level of an asset created using a curated model database in which a user may set a level of detail, or concentration, to be included in the scene.
  • a level of detail or concentration
  • FIG. 27 illustrates a zoomed-in level of an asset created using a curated model database in which a user may set a level of detail, or concentration, to be included in the scene.
  • cellular mega-structures such as the endoplasmic reticulum and endosymbiotic organelles are depicted.
  • FIG. 28 shows a visual product in which a user has increased a level of complexity and the cell is depicted with substantially all macromolecules present.
  • a key is set off to the side to show, for example, how color-coding may be used.
  • the key can indicate that all nucleic acids are red, all proteins are blue, all lipids are grey, and all carbohydrates are brown. Additionally, the key can be used to toggle the appearance of certain classes of macromolecule on and off. Since each macromolecule is represented using a rigged model 121 built on data that is sourced from primary scientific research, the actions and interactions depicted within the visual cell will be scientifically accurate.
  • FIG. 29 shows a zoomed-in view from The Visual Cell that includes a close-up cross section of a mitochondrion within the cytoplasmic environment that it inhabits.
  • the circular mitochondrial genome can be viewed by the user as can the functioning components of the electron transport chain.
  • a user could choose to zoom in on, for example, an ATPase to observe how that piece of cellular machinery exploits a proton concentration gradient to cause rotation, which mechanical energy is transferred by phosphorylating ADP to ATP, a universal store and source of energy.
  • the visual cell can be used to understand how our endosymbiotic history enables aerobic respiration to provide the energy we use to live and grow.
  • the Visual Cell tailors the materials that it presents to its users in several different ways.
  • Each module within The Visual Cell database is 'rated' by curricular and complexity level. Therefore while a set of 20 modules might cover a specific topic like 'Biomembranes', only 5 of them would be rated/tagged for inclusion in an AP-Bio learning path/classroom, 12 would be tagged for use in Introductory Biology and the remaining 3 would be advanced modules (for advanced students or scientists).
  • subsets of widgets can also be grouped according to another cross-cutting themes such as evolution or topics that align themselves with educational standards. For example, an instructor may want to assemble a set of widgets that focuses on evolutionary aspects of biology and thereby find and assign widgets that are tagged with this underlying theme, but belong to many different specific mini-curriculum topics (i.e.
  • the Visual Cell i.e. a 'learning path' through the material of that mini-curriculum.
  • This order of widgets (as noted above it could be 5 or 12 or 20 depending on level of the audience), however, can also be customized by the instructor.
  • the UI of The Visual Cell allows instructors to inspect a library of all widgets within a mini-curriculum (represented with small icons and titles), individually drag-and-drop these widgets onto a custom learning path template and then assign this new custom order to their virtual classroom.
  • Customization in The Visual Cell can also be found at the level of individual modules and their functionalities. For example, modules that allow students to create their own custom 'recipes' for assembling a molecular landscape image are unique in that the output of these activities are completely custom to each student (no 2 images generated by these modules is likely to ever be the same since the student recipe drives a simulation to position molecular components into the 3D scene and resulting image).
  • Visual Cell collects various types of materials to embed within their own digital study portfolio.
  • the study portfolio is also used by the student to write notes and associate them with collected materials as preparation for class tests ('collected materials' in The Visual Cell might be a snapshot from an instructional movie, or a custom image or model created within an assessment module or the result of a simulation launched by the student).
  • Instructors are able to review students' digital study portfolios and make edits (or suggest edits for the students to make).
  • the creation and review of these study portfolios is a valuable assessment activity in itself since it gives instructors a good sense of what the student understands about the material as well as any misconceptions they may have. Since each student's activities, associated notes and assessments is unique to that student, the study portfolio is another example of a tailored experience within The Visual Cell. Additionally, visual assessment may be included in The Visual Cell and its component modules.
  • Visual assessment paradigms within the digital modules of The Visual Cell leverage the richness of the varied forms of visual media that we use for instructional purposes. They offer a unique opportunity to reuse and cast in a different light visual activities driven by students that tie back to the instructional materials and therefore offer a more consistent experience to the student.
  • Example of such visual assessment modules may include any of the following.
  • the interactive labeling of a figure or diagram by a student (a visual similar or identical to those they have previously encountered in an instructional module but that they are now challenged to label, annotate and/or comment on).
  • a 3D interactive module can ask students to manipulate and orient a 3D model in space (whether it be a molecule, tissue, organ, organism or other instruments). Assessment requires students to select specific angles that showcase certain characteristics of the model (in the case of a molecule/enzyme, for ex, the assessment may require the student to orient the molecular such that the active site is facing the camera).
  • One possible module provides free-hand annotations of existing visuals (i.e. 'circle the cytoplasmic domain of this protein').
  • 'guided sketching' activities the freedom of such assessment provides instructors with an even broader window on what students are thinking (see, e.g., the results of the 'Picturing to Learn' NSF project led by Felice Frankel).
  • visual assessment harnesses the benefits of visual thinking in students. It broadens the scope of assessments because many more concepts (and misconceptions) can be gleaned from students through the use of a multitude of visuals - whether static, interactive or custom-created by students themselves.
  • the richness of visual assessment activities also has the potential to the paradigm of 'learn, learn, learn, learn, learn, asses' model to a demonstratively more effective type of instruction (in terms of understanding and long-term retention) which follows the 'learn, asses, learn, asses, learn, asses' model. The latter begins to blur the distinction between learning and assessment phases as a result of the rich experience provided by creative visual assessments.
  • BIOMEMBRANES visual mini curriculum (MEDIA SPEC)
  • FIG. 30 illustrates a detail of a membrane.
  • FIG. 31 shows an H-bond network in ice, bulk water and around a fatty chain.
  • Narrated split-screen movie that compares micelle formation using a single-chain phospholipid (PDC, dodecyl-phospho -choline ) and bilayer formation using a double-chain phospholipid (POPC, palmitoyl-oleoyl-phosphatidyl-choline).
  • PDC single-chain phospholipid
  • POPC palmitoyl-oleoyl-phosphatidyl-choline
  • FIG. 34 shows 1 phospholipid in a leaflet on the left versus 1 Archaeal lipid on the right.
  • FIG. 35 illustrates a structural difference at increased temperatures.
  • Membranes are fluid mosaics of lipids and proteins - almost every protein in the membrane is laterally 'sensing' another protein that is only a few lipid radii away. Extra- membranous domains of peripheral and integral proteins lead to significant crowding of membrane surfaces, as well as the contribution of carbohydrates (glycosylation).
  • FIG. 36 illustrates a lipid bilayer
  • heterokaryons although they eventually completely overlapped. This is due to the fact that the anti-human stain was using sera raised against whole human cells, whereas antibody used to stain the mouse cells/antigens was specific to the H2 antigen (i.e. MHC) which is now known to exist in clusters and probably has reduced mobility.
  • MHC H2 antigen
  • This unique module leverages a database of curated molecular models
  • the module randomly assigns (optionally adaptively based on past performance in certain areas related to membrane diversity) to the student a type of membrane to model (e.g., an animal membrane, a plant cell membrane, make a bacterial membrane, an erythrocyte membrane from someone who is in the A blood group, etc).
  • a type of membrane to model e.g., an animal membrane, a plant cell membrane, make a bacterial membrane, an erythrocyte membrane from someone who is in the A blood group, etc.
  • This image becomes the basis for either a live flipped classroom activity or social media-based virtual/forum-based activity - whereby students, guided by the instructor, critique each other' s imagery for veracity/accuracy.
  • This interactive figure is a select visual catalog of structures/folds that nature has evolved to span the lipid bilayer.
  • Default state of figure shows a single horizontal cross-section of a membrane with many proteins embedded/lined-up next to one another - the graphic style highlights the secondary structure of the transmembrane portion of each and showcases the structural diversity of folds used to span the membrane. By mousing-over each structure, student reveals (in close-up if necessary) the key hydrophobic side chains that enable these TM domains.
  • a patient of known blood group enters the ER, and in need of a blood transfusion - ER has just received blood but the label has fallen off the bag and blood is now of unknown blood group.
  • the goal of the game is to test and identify the blood group for the blood sample in order to start the transfusion and save the patient.
  • FIG. 37 illustrates types of transport
  • hemolysin A (beta sheet-based), cytolysin (alpha-helical)
  • FIG. 38 shows an overview of membrane-enclosed organelles.
  • Membrane curvature Examples of how certain proteins (i.e. B AR-domains and others) preferentially bind curved membranes and also stabilize them (relevance to the function and maintenance of the endomembrane system).
  • FIG. 39 shows organelles and molecular actors relevant to endomembrane transport system (including endo-, exocytosis and vesicle transport)
  • FIG. 40 shows a hemi-fusion intermediate
  • Additional modules that may be optionally include in BIOMEMBRANES visual mini curriculum
  • MDR Multi-Drug Resistance

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • Processing Or Creating Images (AREA)

Abstract

La présente invention concerne des procédés et des systèmes qui peuvent être utilisés pour communiquer visuellement un concept scientifique par construction d'un produit de visualisation à l'aide des actifs préfabriqués pour représenter des entités dans la visualisation. Le produit de visualisation comprend au moins un actif numérique qui transporte visuellement au moins une partie d'un concept scientifique. L'actif numérique comporte des données représentant une structure et un appareil qui définit la dynamique d'animation destinée à la structure et est adapté en fonction d'un niveau d'éducation d'un auditoire.
PCT/US2015/021721 2014-03-20 2015-03-20 Systèmes et procédés destinés à la fourniture d'un produit de visualisation WO2015143303A1 (fr)

Applications Claiming Priority (12)

Application Number Priority Date Filing Date Title
US14/220,702 US20150269870A1 (en) 2014-03-20 2014-03-20 Visual cell
US14/220,730 US20150269855A1 (en) 2014-03-20 2014-03-20 Systems and methods for interacting with a visual cell
US14/220,647 2014-03-20
US14/220,616 US20150269765A1 (en) 2014-03-20 2014-03-20 Systems and methods for providing a visualization product
US14/220,730 2014-03-20
US14/220,718 US20150269849A1 (en) 2014-03-20 2014-03-20 Systems and methods for use of digital assets
US14/220,718 2014-03-20
US14/220,702 2014-03-20
US14/220,647 US20150269763A1 (en) 2014-03-20 2014-03-20 Curated model database
US14/220,616 2014-03-20
US14/662,888 2015-03-19
US14/662,888 US9875567B2 (en) 2014-03-20 2015-03-19 Systems and methods for modeling

Publications (1)

Publication Number Publication Date
WO2015143303A1 true WO2015143303A1 (fr) 2015-09-24

Family

ID=52815312

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2015/021721 WO2015143303A1 (fr) 2014-03-20 2015-03-20 Systèmes et procédés destinés à la fourniture d'un produit de visualisation

Country Status (1)

Country Link
WO (1) WO2015143303A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019079251A1 (fr) * 2017-10-18 2019-04-25 President And Fellows Of Harvard College Modélisation de structures moléculaires
CN111696184A (zh) * 2020-06-10 2020-09-22 上海米哈游天命科技有限公司 骨骼蒙皮融合确定方法、装置、设备和存储介质
CN117392294A (zh) * 2023-12-08 2024-01-12 广东咏声动漫股份有限公司 一种动画文件检查修复方法、系统、电子设备及存储介质
CN118227093A (zh) * 2024-04-08 2024-06-21 苏州科技大学 一种基于大数据的图像设计视觉传达系统及其方法

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1274060A1 (fr) * 2001-07-06 2003-01-08 Saga University Système et méthode de production de matière à enseigner
US20060109274A1 (en) 2004-10-28 2006-05-25 Accelerated Pictures, Llc Client/server-based animation software, systems and methods
US20070016853A1 (en) * 2005-07-14 2007-01-18 Molsoft, Llc Structured documents and systems, methods and computer programs for creating, producing and displaying three dimensional objects and other related information in those structured documents
WO2008154487A1 (fr) * 2007-06-07 2008-12-18 Monarch Teaching Technologies, Inc. Système et procédé pour générer des leçons visuelles personnalisées
US20090091563A1 (en) 2006-05-05 2009-04-09 Electronics Arts Inc. Character animation framework
US20090177455A1 (en) * 2007-12-14 2009-07-09 University Of North Dakota Method for animating chemical mechanisms
US20090295793A1 (en) 2008-05-29 2009-12-03 Taylor Robert R Method and system for 3D surface deformation fitting
US7782324B2 (en) 2005-11-23 2010-08-24 Dreamworks Animation Llc Non-hierarchical unchained kinematic rigging technique and system for animation
US20110098113A1 (en) 2009-10-28 2011-04-28 Robert Lanciault Systems and methods for portable animation rigs
US8253745B1 (en) 2008-03-18 2012-08-28 Pixar Systems and methods for creating and using modeless animation manipulation rigs
US20130222265A1 (en) * 2012-02-24 2013-08-29 Robin Young Smith Systems, Methods, and Apparatus for Drawing Chemical Structures Using Touch and Gestures
US20130235046A1 (en) 2012-03-07 2013-09-12 Unity Technologies Canada Inc. Method and system for creating animation with contextual rigging
US8624898B1 (en) * 2009-03-09 2014-01-07 Pixar Typed dependency graphs

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1274060A1 (fr) * 2001-07-06 2003-01-08 Saga University Système et méthode de production de matière à enseigner
US20060109274A1 (en) 2004-10-28 2006-05-25 Accelerated Pictures, Llc Client/server-based animation software, systems and methods
US20070016853A1 (en) * 2005-07-14 2007-01-18 Molsoft, Llc Structured documents and systems, methods and computer programs for creating, producing and displaying three dimensional objects and other related information in those structured documents
US7782324B2 (en) 2005-11-23 2010-08-24 Dreamworks Animation Llc Non-hierarchical unchained kinematic rigging technique and system for animation
US20090091563A1 (en) 2006-05-05 2009-04-09 Electronics Arts Inc. Character animation framework
WO2008154487A1 (fr) * 2007-06-07 2008-12-18 Monarch Teaching Technologies, Inc. Système et procédé pour générer des leçons visuelles personnalisées
US20090177455A1 (en) * 2007-12-14 2009-07-09 University Of North Dakota Method for animating chemical mechanisms
US8253745B1 (en) 2008-03-18 2012-08-28 Pixar Systems and methods for creating and using modeless animation manipulation rigs
US20090295793A1 (en) 2008-05-29 2009-12-03 Taylor Robert R Method and system for 3D surface deformation fitting
US8624898B1 (en) * 2009-03-09 2014-01-07 Pixar Typed dependency graphs
US20110098113A1 (en) 2009-10-28 2011-04-28 Robert Lanciault Systems and methods for portable animation rigs
US20130222265A1 (en) * 2012-02-24 2013-08-29 Robin Young Smith Systems, Methods, and Apparatus for Drawing Chemical Structures Using Touch and Gestures
US20130235046A1 (en) 2012-03-07 2013-09-12 Unity Technologies Canada Inc. Method and system for creating animation with contextual rigging

Non-Patent Citations (19)

* Cited by examiner, † Cited by third party
Title
ALBERTS; JOHNSON: "Molecular Biology of the Cell", 2014, GARLAND SCIENCE, pages: 1464
BERMAN ET AL., THE PROTEIN DATA BANK, NUCL ACIDS RES, vol. 28, no. 1, 2000, pages 235 - 242
CHAPPELL ET AL.: "Crystal structure of reovirus attachment protein sigmal reveals evolutionary relationship to adenovirus fiber", EMBO J, vol. 21, 2002, pages 1 - 11
COOPER; HAUSMANN: "Cell: A Molecular Approach", 2013, SINAUER ASSOCIATES, INC., pages: 832
DAVID A. D. GOULD: "Complete Maya Programming: An Extensive Guide to MEL and the C++ API", 2003, MORGAN KAUFMANN
FRYE; EDIDIN: "The rapid intermixing of cell surface antigens after formation of mouse-human heterokaryons", JOURNAL OF CELL SCIENCE, vol. 7, 1970, pages 319 - 335
GOULD: "Complete Maya Programming: An Extensive Guide to MEL and the C++ API", 2003, MORGAN KAUFMANN, pages: 528
GREEN; SAMBROOK: "Molecular Cloning: a Laboratory Manual", 2012, COLD SPRING HARBOR LABORATORY PRESS, pages: 2028
KARP: "Cell and Molecular Biology: Concepts and Experiments", 2013, WILEY, pages: 864
KIM ET AL., NUCL AC RES, vol. 29, 2011, pages D451 - 5
LIU; KARIMI: "High-throughput modeling and analysis of protein structural dynamics", BRIEF BIOINFORM, vol. 8, no. 6, 2007, pages 432 - 45
O'HAILEY: "Rig it Right! Maya Animation and Rigging Concept", 2013, FOCAL PRESS, pages: 280
O'HAILEY: "Rig it Right! Maya Animation and Rigging Concepts", 2013, FOCAL PRESS, pages: 280
PALAMAR; KELLER: "Mastering Autodesk Maya", 2011, WILEY PUBLISHING, pages: 950
PENNARUM ET AL.: "Playing the DISC: turning on the TRAIL death receptor-mediated apoptosis in cancer", BIOCHIM BIOPHYS ACTA, vol. 1805, no. 2, 2010, pages 123 - 40
PETERS ET AL.: "A microfluorimetric study of translational diffusion in erythrocyte membranes", BIOCHIMICA ET BIOPHYISCA ACTA--BIOMEMBRANES, vol. 367, no. 3, 1974, pages 282 - 294
SHARPE ET AL.: "silico: 3D Animation and Simulation of Cell Biology with Maya and MEL", 2008, ELSEVIER MORGAN KAUFMAN, pages: 622
SINGER; NICHOLSON: "The fluid mosaic model of the structure of cell membranes", SCIENCE, vol. 175, no. 4023, 1972, pages 720 - 31
WHITFORD: "Proteins: Structure and Funtion", 2005, WILEY, pages: 542

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019079251A1 (fr) * 2017-10-18 2019-04-25 President And Fellows Of Harvard College Modélisation de structures moléculaires
CN111696184A (zh) * 2020-06-10 2020-09-22 上海米哈游天命科技有限公司 骨骼蒙皮融合确定方法、装置、设备和存储介质
CN111696184B (zh) * 2020-06-10 2023-08-29 上海米哈游天命科技有限公司 骨骼蒙皮融合确定方法、装置、设备和存储介质
CN117392294A (zh) * 2023-12-08 2024-01-12 广东咏声动漫股份有限公司 一种动画文件检查修复方法、系统、电子设备及存储介质
CN117392294B (zh) * 2023-12-08 2024-04-09 广东咏声动漫股份有限公司 一种动画文件检查修复方法、系统、电子设备及存储介质
CN118227093A (zh) * 2024-04-08 2024-06-21 苏州科技大学 一种基于大数据的图像设计视觉传达系统及其方法

Similar Documents

Publication Publication Date Title
US9875567B2 (en) Systems and methods for modeling
US20150269870A1 (en) Visual cell
Elmunsyah et al. Interactive learning media innovation: utilization of augmented reality and pop-up book to improve user’s learning autonomy
Johnson et al. A guide to the visual analysis and communication of biomolecular structural data
Doak et al. Peppy: a virtual reality environment for exploring the principles of polypeptide structure
WO2019010426A1 (fr) Interface utilisateur pour système de gestion de l'apprentissage
Ali et al. Interactive laboratories for science education: A subjective study and systematic literature review
WO2015143303A1 (fr) Systèmes et procédés destinés à la fourniture d'un produit de visualisation
US20150269849A1 (en) Systems and methods for use of digital assets
Shaer et al. Designing reality-based interfaces for experiential bio-design
Joseph et al. Instructional design for learning path identification in an e-learning environment using felder-silverman learning styles model
Blattgerste et al. Train AR: A scalable interaction concept and didactic framework for procedural trainings using handheld augmented reality
Bredeweg et al. Learning with interactive knowledge representations
Gutiérrez et al. Networked analysis of a teaching unit for primary school symmetries in the form of an e-book
McGill Designing instructional science visualizations in the trenches: Where research meets production reality
US20150269763A1 (en) Curated model database
US20150269855A1 (en) Systems and methods for interacting with a visual cell
US20160260232A1 (en) Systems and methods for creation of visualizations
Zamora-Antuñano et al. Methodology for the development of augmented reality applications: MeDARA. Drone Flight Case Study
Ahmed et al. VR-Based Visualization for Biological Networks
Nascimento Júnior et al. Enhancing AI Responses in Chemistry: Integrating Text Generation, Image Creation, and Image Interpretation through Different Levels of Prompts
Zarei et al. A novel approach of multimedia instruction applications in engineering education
US20150269765A1 (en) Systems and methods for providing a visualization product
Span et al. Protein structure in context: the molecular landscape of angiogenesis
Singla et al. A new visual design language for biological structures in a cell

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15715037

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 12/01/2017)

122 Ep: pct application non-entry in european phase

Ref document number: 15715037

Country of ref document: EP

Kind code of ref document: A1