CN115858658A - A family genealogy system and its processing method - Google Patents

Abstract

The invention discloses a family genealogy system and a processing method thereof. The method includes the following steps: obtaining personal information of a natural person and the family relationship among family members based on an input interface or interface; establishing a family tree of a natural person based on the family relationship of the natural person, And establish the relationship between the family tree and the personal information of each family member; create virtual display data of the family tree based on the family tree data and the personal information of the family members; extract the first application data of the corresponding type from the family tree data based on the application type, and based on the The first application data extracts virtual display data from the family tree virtual display data, and generates the first virtual display data displayed in AR, VR or MR; and based on a user request corresponding to an application type, uses AR, VR or MR The first virtual display data is displayed in a manner. The present invention solves the technical problem of the current lack of natural person blood/family relationship data records.

Description

A family genealogy system and its processing method

technical field

The present invention relates to the technical field of information collection and processing, and also relates to the technical field of MR (Mixed Reality), in particular to a family genealogy system and its processing method, a server, a computer-readable storage medium and a computer program product.

Background technique

The current Internet is full of massive amounts of information. The content expressed by the same piece of data may be precious information to some people, but it may be useless junk information to others. People have been exploring and trying ways to obtain information efficiently, one of which is to build various types of databases. For example, in order to facilitate doctors to accurately understand the patient's past medical history, medical experience and medication, each hospital has a patient's medical record database, and with the development of information technology, medical records between hospitals within a medical system Databases can be interconnected and shared. Another example is the geographic location database that supports intelligent transportation systems and location-based services. Another example is a genetics database that stores genetic data (such as genes, gene products, variants, traits, etc.), so that users can retrieve genetic data, add genetic data, and extract information. Or government agencies, such as civil affairs departments, have databases that store people's marriage relationships, etc. However, the information provided by these databases is usually specific to a certain aspect and independent of each other. For a natural person with social attributes, the information about the natural person includes various aspects, such as medical information, genetic disease information, geographic location information, marital status information, etc., and these information may be distributed in different systems. Repeated, but not comprehensive. In addition, the information of each natural person is separated in various systems and databases, and the natural person data recorded in these systems and databases rarely have data on blood relationship. For example, the family relationship between natural persons can only be queried in the population and household registration management system, and this family relationship between natural persons is usually limited to one family in one household registration record. For two natural persons who are not in the same household registration record, even if the blood relationship is very close, the blood relationship between the two cannot be directly obtained from the database, and there is no way to know the situation of two or more generations. As for other systems, such as databases in hospital systems, data on blood relationship between people is rarely recorded.

Contents of the invention

In view of this, the embodiments of the present invention provide a family tree system and its processing method, server, computer-readable storage medium and computer program products, which are used to solve the current technical problem of lack of blood relationship/family relationship data records between natural persons.

In order to solve the above technical problems, according to one aspect of the present invention, the present invention provides a family tree system processing method, comprising the following steps:

Obtain the personal information of natural persons and the family relationship among family members based on the input interface or interface;

Establish a family tree of a natural person based on the family relationship of the natural person, and establish the relationship between the family tree and the personal information of each family member;

Create family tree virtual display data based on family genealogy data and family member personal information, said virtual display data including one or more of one or more virtual scenes, virtual models and personal information labels of one or more family members;

Based on the application type, extract the corresponding type of first application data from the family tree data, and extract one or more of the corresponding virtual scene virtual model and personal information label from the family tree virtual display data based on the first application data, and merged into the first virtual display data displayed in AR, VR or MR; and

Based on a user request corresponding to an application type, the first virtual display data is displayed in an AR, VR or MR manner.

According to another aspect of the present invention, the present invention provides a family tree system, which includes: a data entry module, a data association module, a virtual scene creation module, a first application processing module and a presentation module, wherein the data entry module It is used to obtain the personal information of natural persons and the family relationship between family members based on the input interface or interface; the data association module is connected to the data entry module, and the family tree of the natural person is established based on the family relationship of the natural person, and the family tree and the family tree are established The relationship between the personal information of each family member; the virtual scene creation module is configured to create virtual display data of the family tree based on the family tree data and the personal information of the family members, and the virtual display data includes one or more of one or more family members One or more of a virtual scene, a virtual model, and a personal information tag; the first application processing module extracts the corresponding type of first application data from the family genealogy data based on the application type, and extracts the corresponding type of first application data from the family tree data based on the first application data. One or more of the corresponding virtual scene virtual model and personal information tags are extracted from the virtual display data of the family tree, and are fused into the first virtual display data displayed in AR, VR or MR; the display module and the The first application processing module is connected to display the first virtual display data in AR, VR or MR based on a user request corresponding to an application type.

According to another aspect of the present invention, the present invention provides a server, including a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the aforementioned family tree system processing method is realized.

According to another aspect of the present invention, the present invention provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the aforementioned family tree system processing method is implemented .

According to another aspect of the present invention, the present invention provides a computer program product, which includes computer program instructions, and when the computer program instructions are executed by a processor, the aforementioned family tree system processing method is implemented.

The family genealogy system and its processing method provided by the present invention can not only record the blood relationship/family relationship between natural persons, but also provide various types of data related to natural persons, the data types are rich and interrelated. The system and its processing method described in the present invention display data in AR, VR or MR mode when presenting data to people, with strong visual effect and better experience.

Description of drawings

In order to more clearly describe the implementation of the embodiments of the present invention, the following briefly introduces the drawings in the embodiments of the present invention.

FIG. 1 is a schematic diagram of an AR system architecture based on a server and a terminal device according to an embodiment of the present invention.

Figure 2 is a schematic diagram of a virtual-real fusion image using a mobile phone APP for AR navigation.

Fig. 3 is a flowchart of a processing method of a family tree system according to an embodiment of the present invention.

Fig. 4A is a flowchart of a method for establishing a family tree according to an embodiment of the present invention.

FIG. 4B is a graphical representation of a tree structure of a family tree according to an embodiment of the present invention.

Fig. 5 is a flowchart of a method for creating virtual display data of a family tree according to an embodiment of the present invention.

Fig. 6 is a flowchart of a method for obtaining first application data according to an embodiment of the present invention.

Fig. 7 is a flowchart of a process for a user to create a family tree according to an embodiment of the present invention.

FIG. 8 is a flowchart of a process for a user to view family tree data according to an embodiment of the present invention.

Fig. 9 is a functional block diagram of a family genealogy system according to an embodiment of the present invention.

Fig. 10 is a functional block diagram of a family tree system according to another embodiment of the present invention.

Fig. 11 is a functional block diagram of a family tree system according to yet another embodiment of the present invention.

Fig. 12 is a functional block diagram of the C-S architecture of the family genealogy system according to an embodiment of the present invention.

Fig. 13 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Fig. 14 is a schematic diagram of a software structure of an exemplary terminal device according to an embodiment of the present invention.

Detailed ways

The principle and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that the purpose of providing these embodiments is to make the principle and spirit of the present invention more clear and thorough, so that those skilled in the art can better understand and realize the principle and spirit of the present invention. The exemplary embodiments provided herein are only some, not all, embodiments of the present invention. Based on the implementation manners herein, all other implementation manners obtained by persons of ordinary skill in the art without making creative efforts fall within the protection scope of the present invention.

Those skilled in the art know that the embodiments of the present invention can be implemented as a system, device, device, method, computer-readable storage medium or computer program product. Therefore, the present invention can be embodied in at least one of the following forms: complete hardware, complete software, or a combination of hardware and software. According to the specific implementation mode of the present invention, the present invention claims a family tree system, a processing method and system, a terminal device, a server, a computer-readable storage medium and a computer program product.

In this article, terms such as first, second, and third are only used to distinguish one entity (or operation) from another entity (or operation), and are not intended to require or imply that these entities (or operations) operations) in any order or relationship.

Embodiments of the present invention are applicable to servers and terminal devices. Please refer to FIG. 1 , which schematically shows a schematic diagram of an AR system architecture based on a server and a terminal device. The AR system architecture includes a server 10 and several terminal devices 20 . In some examples, the terminal device 20 is an AR device, which may be a dedicated AR device, such as smart wearable electronic devices such as head-mounted displays (HMD), smart gloves, and clothing. In some examples, the terminal device 20 may be a general AR device, such as a mobile phone, a portable computer, a notebook computer, a tablet computer, a virtual reality (Virtual Reality, VR) device, a vehicle device, a navigation device, a game device, and the like.

Taking AR helmet or AR glasses as an example, the head-mounted display, machine vision system, mobile computer, etc. can be integrated into a device that can be bound and worn. Head, the device can transmit augmented reality information to the display or project it into the user's eyeballs, thereby enhancing the user's visual immersion. In some examples, the AR device also has a camera, which may be a wide-angle camera, a telephoto camera, or a structured light camera (also called a point cloud depth camera, a 3D structured light camera or a depth camera). Among them, the structured light camera is based on 3D vision technology, which can obtain the plane and depth information of the object. The structured light camera can project light with certain structural characteristics onto the object to be photographed through the near-infrared laser, and then collect the reflected light by the infrared camera, which is processed by the processor chip. Position and depth information to render 3D images. Ordinary terminal devices such as mobile phones present two-dimensional images, which cannot display the depth of different positions on the image. The structured light camera can be used to capture and obtain 3D image information data, that is, not only the color information of different positions in the image can be obtained, but also Depth information of different locations can be used for AR ranging. Of course, ordinary terminal devices can also collect 2D images based on optical cameras and combine deep learning algorithms to obtain depth information of 2D images, and finally can also present 3D images.

In some examples, the terminal device 20 is installed with software or application APP with AR function. The server 10 may be a management server or an application server of the software or APP. The server 10 can be one server, or a server cluster composed of multiple servers, or a cloud server or cloud server. Modules with networking functions, such as Wireless-Fidelity (Wifi) modules, Bluetooth modules, 2G/3G/4G/5G communication modules, etc., are integrated in the terminal device 20 to connect to the server 10 through the network.

Exemplarily, the user can log in the user account through the APP installed in the mobile phone, and the user can also log in the user account through the software installed in the AR glasses.

Taking an APP with AR navigation function as an example, the APP can have, for example, high-precision map navigation capabilities, environmental understanding capabilities, and virtual-real fusion rendering capabilities. The APP can report the current geographic location information to the server 10 through the terminal device 20. The location information provides users with AR navigation services. Exemplarily, taking the terminal device 20 as a mobile phone as an example, in response to the operation of the user starting the APP, the mobile phone can start the camera to collect images of the real environment, and then perform AR enhancement on the real environment images collected by the camera through the system. Integrating or superimposing rendered AR effects (such as navigation route signs, road names, business information, advertisement display, etc.), displaying virtual and real images on the mobile phone screen.

Figure 2 schematically shows a virtual-real fusion image using mobile APP for AR navigation, in which the indicator arrow of AR navigation is superimposed on the real road surface and space in the picture, and the electronic resources promoted by merchants use parachutes to carry gift boxes The form floats at a given position in space.

Embodiments of the invention relate to terminal devices and/or servers. The principle and spirit of the present invention will be explained in detail below through several exemplary embodiments or representative implementations.

Fig. 3 is a flowchart of a family tree system processing method according to an embodiment of the present invention, and the system processing method includes the following steps:

Step S11, based on the input interface or interface, the personal information of the natural person and the family relationship among family members are obtained.

Step S12, establishing the family tree of the natural person based on the family relationship of the natural person, and establishing the association relationship between the family tree and the personal information of each family member.

Step S13, create family tree virtual display data based on family genealogy data and family member personal information, the virtual display data includes one or more of one or more virtual scenes, virtual models and personal information labels of one or more family members many.

Step S14, extracting the corresponding type of first application data from the family tree data based on the application type, extracting virtual display data from the family tree virtual display data based on the first application data, and merging them to display in AR, VR or MR The first virtual display data of .

Step S15 , displaying the first virtual display data in AR, VR or MR based on a user request corresponding to an application type.

In step S11, the family tree system of the present invention provides an input interface for users to input data through the input interface. The data entered includes personal information of natural persons and family relationships among family members. The natural person personal information includes some basic personal information, such as name, age, height, weight, gender, place of birth, date of birth, education experience, work experience, marital status, latest contact information, etc. After entering the personal information of multiple family members, it is also necessary to determine the basic family relationship among family members, which can also be divided into legal relationship and biological relationship. Relationships in the legal sense, such as the relationship of biological children and stepchildren in a legal marriage relationship, the relationship of biological children who have divorced from a legal marriage relationship, etc. The biological relationship includes, for example, a biological relationship with a blood relationship, an adoption relationship without a blood relationship, and the like.

In an embodiment, the input interface may be provided by a client installed in a user terminal device. The client provides various interfaces for interacting with users to fulfill various interaction requirements. Wherein, the input interface may be displayed by the display of the user terminal, and a keyboard, a microphone, etc. of the user terminal may be used as data input tools. At this time, the user can input data in the form of text, image, audio, video, etc. one by one according to the data items given in the input interface through keyboard operation, or input corresponding data through voice input. For example, when entering personal information, information such as name, age, height, weight, gender, place of birth, date of birth, etc. can be entered in text through the keyboard, or can be entered in voice through the microphone. It is also possible to upload images, audio, and video data through keyboard operations, such as photos of a natural person in various periods, life scene videos, audio, etc.

The entry interface can also be presented to the user through the user terminal in the form of AR or VR. At this time, the user can use the camera in the user terminal as an input tool to scan, identify and input some documents, pictures, etc., and the process of scanning and identifying can be assisted by user gestures. For example, the user selects the document to be entered from the current pile of text documents through AR glasses and gestures, such as a birth certificate, starts the camera to scan, and recognizes the text in it, and then obtains the required data through semantic analysis. The client sends the obtained data to the server of the system.

In addition, when entering data, it can also connect with other third-party systems through the interface to obtain relevant data from other third parties, for example, connect with the hospital system, and receive medical treatment and medical record data of a specific person from the hospital's medical record database.

In step S12, after the server of the system receives the data sent by the client, it establishes the relationship between the family tree and the personal information of each family member through processing. As shown in FIG. 4A, FIG. 4A is a flowchart of a method for establishing a family tree according to an embodiment of the present invention. Include the following steps:

Step S121, receiving natural person personal information and family relationship information among family members.

In step S122, the information is extracted, classified, and stored as node data. In order to facilitate data extraction and retrieval, the present invention sets multiple attributes for personal information, and each attribute includes one or more classified fields. For example, the attributes mentioned are, for example, general attributes, including classification fields such as date of birth, gender, ID card information, and the like. The attributes are also, for example, physiological attributes, including classification fields such as height, weight, blood type, and medical record data. The attribute is also, for example, a geographic attribute, including classification fields such as place of birth, all previous home addresses, addresses of work units, and the like. The attributes are also, for example, social attributes, including classification fields such as school information during the education process, academic degree information, occupation information after work, and the like. The attribute is also, for example, a communication attribute, including classification fields such as phone number, social account number, and the like. In addition, the various data above correspond to time information, such as height data in physiological attributes. When there are multiple height data, each piece of height data corresponds to the time when the height data was measured. Another example is that each piece of data in the medical record data corresponds to the visit time. Another example is the time of going to kindergarten, the time of entering elementary school... the time of college graduation, the time of each career entry, etc. In addition, the system creates a personal file for each natural person to store personal information, which is stored according to attributes and categories. In this step, when the newly entered data is received, all the information of each natural person is firstly extracted and classified according to the personal identification from the currently received data, and the personal identification is, for example, name. Then extract the content of relevant fields from all the information of each natural person according to each attribute and corresponding classification field. It is then categorized and stored in the personal file of the natural person. When there is a personal file with the same name in the system, the exact personal file can be found by supplementing with place of birth, date of birth, names of parents, etc. Among them, each natural person is regarded as a node, and the data in his personal file is regarded as node data. Wherein, in order to facilitate displaying different information of a member in the family tree to different users, the data in the personal file is provided with multiple different display levels, and the display levels correspond to permissions.

Step S123, creating a tree structure of the family tree and associating the node data of each node in the tree structure. After the family relationship among the family members of a family is determined, a tree structure of the family tree is established based on the relationship, and its expression or presentation may be in the form of graphs, tables, and the like. Graphically as shown in Figure 4B. Each family member in the graph uses its recent photo as a node, and of course the 3D model can also be used as a node. And, each node is associated with a personal profile as node data. When the family tree is displayed to users with different permissions, users can view the information of the corresponding display level according to the association between nodes and node data.

In order to make the family tree provided by the present invention applicable to various technologies such as virtualization and augmentation, in step S13, virtual display data of the family tree is created based on the family tree data and personal information of family members. The virtual display data includes a tree diagram of a family tree, such as a two-dimensional plane diagram suitable for augmented reality, and a three-dimensional stereogram suitable for virtual reality and mixed reality. The virtual display data may also include one or more virtual scenes for each family member. The virtual scene corresponds to, for example, a scene of important events in the life of the family member, and the scene may be a static scene or a dynamic scene. The scene features the family member as the protagonist, and also includes some other people, places, backgrounds, etc. as necessary. For example, a birth scene includes the background of the place of birth, the midwife, the mother and the family members. The background of the place of birth, such as the delivery room of a hospital, the rooms and furnishings of a personal house, etc. The virtual scene is another example of a college graduation scene, a wedding scene, a funeral scene, and the like. The virtual display data may also be a virtual model, such as a three-dimensional human body model of a family member at different ages, a three-dimensional model of an item that is important to the family member, and the like. The virtual display data may also be a personal information tag, for example, personal information corresponding to viewing user rights is recorded. This information can be displayed in the form of AR, VR or MR.

Fig. 5 is a flowchart of a method for creating virtual display data of a family tree according to an embodiment of the present invention. In this embodiment, the following steps are specifically included:

Step S131, acquiring node data of each node in the family tree. In one embodiment, the family tree system is provided with a family tree ID for each family tree, and the IDs of the node data (ie, personal files) of each node in the family tree have a corresponding relationship with the family tree ID. Therefore, the node data storage address of each node can be obtained through the genealogy identification, and the corresponding node data is read out from the storage address.

Step S132, extracting events from the node data according to the timeline and generating event description data. The various data recorded in the personal files that store node data have corresponding time, and events that are significant to a natural person are extracted from them according to the timeline, such as birth, enrollment, transfer, moving, university graduation, and every job entry etc., and read the data related to those events from the profile, generating event description data. For example, for the event of birth, you can read the information of the place of birth, such as the city or country, the hospital where you were born or your home; you can also read the date of birth, corresponding to year, month, and day; if possible, you can also read Birth health records can be read, such as height, weight, vaccination information, whether premature, full-term and other information. Collect these information to get the event description data, such as "born in the first hospital of y city on xx, xxxx, full-term, height 51cm, weight 4kg, inoculated with hepatitis B vaccine and BCG". For another example, the event description data obtained for another natural person's birth event is, for example, "born on xx, xx, xx, xxxx, m village, m township, k city, y province, full month". In addition, the description data may also be a piece of video or audio intercepted from the input data.

Step S133, determine the corresponding display virtual element based on the event description data. The display virtual element may be a three-dimensional model, a segment of video, a picture, or audio. The three-dimensional models are, for example, three-dimensional models of various people, mountain models corresponding to different seasons, part of urban street models representing cities, a section of river model, field models representing plains, various types of house models, object models, etc. . The image is, for example, a representative video or audio. Taking the aforementioned birth event as an example, the corresponding display virtual elements include: city, hospital, baby, delivery room, mother, and the season is winter, night, and baby's cry.

Step S134, searching and obtaining the display virtual element from the resource database. Wherein, various display virtual elements are stored in the resource database, such as the aforementioned three-dimensional model, video, picture or audio. Using the aforementioned birth event as an example, get a 3D model of the baby, mother and delivery room, a 3D model of a city street representing a winter city, a 3D model of a house with a hospital logo, covered in snow, snowflakes and audio of the baby crying .

Step S135, combining the display virtual elements according to the event attributes to obtain virtual display data. Combining the above-mentioned various 3D models into multiple different sub-scenes, and connecting multiple sub-scenes together through transition effects in chronological order, and then obtaining a complete scene, such as the sub-scene of a night city street covered with snowflakes; The sub-scene of the appearance of the hospital covered with snowflakes; including the sub-scene of the baby, the mother and the delivery room; the three sub-scenes from the street, the hospital to the delivery room are connected by transition effects, and the snow and the crying audio of the baby are added, so that Get a 1-3 second of a birth scene. In the same way, the virtual display scenes of all events extracted from the node data are obtained, sorted by the timeline, and connected through transition effects to obtain the display data of a node.

Fig. 5 is only an illustration of obtaining virtual display data, and the various scenes described may also be preset scenes, for example, a variety of typical scenes corresponding to various events and corresponding events are stored in the resource database. Event description data, such as graduation scene, birth scene, etc. After the event is obtained in step S132, the typical scene can be obtained from the resource database, and the difference between the generated event description data and the event description data of the typical scene is compared. If the difference between the two is small, the typical scene is used. is large, determine the virtual element that brings the difference, and then obtain the required virtual element from the resource database to replace the original virtual element in the typical scene, so as to obtain the virtual display scene corresponding to the extracted event.

Through the above-mentioned processing process, a virtual display data of a family tree is obtained, which includes all biographical data of each family member. In one embodiment, the genealogy data of each family can be managed by family members with management authority, including regularly inputting data, correcting errors of entered data, and the like. Correspondingly, the present invention adds newly added data to corresponding node data, and generates virtual display data according to specific requirements. For example, when an administrator of a family tree enters a new family member, the present invention determines the position of the new family member in the tree structure of the family tree according to the input data, and creates a personal file for it to store node data, and at the same time Create a virtual display of the added family member data. When the administrator of a family tree modifies or revises the entered data, the original data is replaced based on the modified data, and based on the new data, it is determined whether it needs to modify its position in the tree diagram and whether it needs to modify the virtual display data. For example, when a person's place of birth is modified, the birth scene is changed based on the new place of birth information.

Because the data in a family tree are various and complex, in order to make full use of these data, the system of the present invention is provided with multiple query categories of family tree data, and each query category corresponds to an application type, for example: Geographic changes, family medical history, family life expectancy, family education, family genetic factors, changes in family social status, etc. Therefore, in step S14, the first application data of the corresponding type is extracted from the family tree data based on the preset categories, and the corresponding virtual display data is extracted from the family tree virtual display data based on these first application data, and generated. The first virtual display data displayed in the form of AR, VR or MR is stored in the resource database.

Fig. 6 is a flowchart of a method for obtaining first application data according to an embodiment of the present invention. In this embodiment, in order to obtain the geographical distribution data of a family member at birth, the following steps are included:

Step S141, obtaining the birth events of each node in the family tree.

Step S142, obtaining map data according to the place of birth in the birth event. In one embodiment, the corresponding three-dimensional geographical terrain model is obtained according to the geographic location of each birthplace. In another embodiment, the overall geographical terrain three-dimensional model of the birthplace of the entire family member may also be included. At this point, mark the place of birth of each family member in the 3D model of the overall geographic terrain.

In step S143, the virtual scene corresponding to the birth event is added to the map, and then the first virtual display data of birth geographical distribution is obtained. For example, the virtual scene of the birth event of each family member is fused into a corresponding independent three-dimensional model of geographical terrain, and then each model is distributed on the planar map. Or directly integrate the virtual scene of each family member's birth event into the overall geographic terrain 3D model. The three-dimensional model of the overall geographic terrain is, for example, a three-dimensional overall terrain model of the earth.

According to the application type, various application data conforming to the type are obtained from the family genealogy data, so that the family genealogy data provided by this system is not only a biographical record of family members of a family, but also can provide data support for various industries. For example, this system can provide a family's health data and genetic data. When these data are provided to a hospital or a specific doctor through an interface, it can help doctors conduct medical research to determine potential diseases that patients may have. For example, when a middle-aged patient suffers from short-term memory loss, the doctor can determine the patient according to the genetic law of the disease by querying the health data of his family. The present symptoms are the pre-symptoms of Alzheimer's disease. Another example is when a couple of young people are about to enter into marriage, by querying their family's health data, it can be determined whether both parties may suffer from certain diseases that are not suitable for marriage, such as familial mental illness, epilepsy, etc.

Fig. 7 is a flowchart of a process for a user to create a family tree according to an embodiment of the present invention. In this embodiment, the user uses AR glasses and installs a client of the family tree system, and the AR glasses can transmit data with the server through the network. This embodiment includes the following steps:

Step S11a, the user starts the family tree system client in the AR glasses.

In step S11b, the AR glasses provide the main interface of the client in response to the user's start-up operation on the client of the family tree system.

In step S12a, the user chooses to create an account and enters user information.

In step S12b, the AR glasses receive user information and generate an account creation request based on the account creation items selected by the user.

In step S13b, the AR glasses send an account creation request to the family genealogy system server.

Step S11c, after receiving the account creation request, the server parses out the user information, and determines the category and authority of the user classification.

In step S12c, the server returns a response message to the AR glasses. This includes account information, permissions, and other instructions, among other things.

In step S14b, the AR glasses provide a data entry interface.

In step S13a, the user inputs data instructions. For example, at this time, the user side has some documents including family member information, such as household registration booklet, ID card, medical record book, passport and so on. At this time, the user can determine a specified file through a gesture.

In step S15b, the AR glasses scan and recognize the current file based on the file specified by the gesture and the user's input data instruction. Wherein, in one embodiment, when identifying the content in the document, the scanned image can be image-processed to recognize the characters in the image through the natural language processing model, and then the semantic recognition can be performed to obtain the text content with semantic meaning. The data input in this embodiment is only an example, and the user can also input the content in the file by voice. For example, the user reads the content in the file with audio, and the microphone in the AR glasses records a piece of audio, and then obtains the text content from the current audio through the speech processing model. In addition, the aforementioned natural language processing, voice processing, etc. can also be completed by the server.

In step S16b, the AR glasses send the recognized data to the server. The data includes text content obtained after various processes, or scanned image files or audio files.

Step S13c, the server receives the data sent by the AR glasses, extracts and classifies the data, generates nodes according to the family members in the data and creates personal files for them, and classifies and stores the extracted data into the personal files to form node data .

Step S14c, obtain relevant data of each family member from databases of other systems, and store them in personal files. The databases of other systems are, for example, hospital databases, crime databases in the public security system, student status databases in the education system, marriage registration databases in the Civil Affairs Bureau, etc., so that other relevant information about each family member that the current user has not entered can be obtained. data.

Step S15c, generating a family tree structure based on family members and their relationships.

Step S16c, creating virtual display data of the family tree.

Step S17c, obtaining user terminal equipment information. For example, the terminal device type currently used by the user is analyzed from the request or data sent by the user terminal. In this embodiment, what the server obtains is AR glasses information.

Step S18c, sending the virtual display data of the family tree suitable for AR glasses display to the AR glasses.

In step S17b, the AR glasses display the family tree in an augmented reality manner.

In the foregoing embodiments, when the user creates a family tree account, the server determines that the user is a family tree user and determines that he is an administrator of the family tree. Administrator users can create a sub-account for each of their family members, and specify permissions for each member, such as the viewing permission of only viewing the family tree, and can also specify which information can be viewed and which information cannot be viewed when viewing. Through the permissions set by the administrator user, when a family member logs in to the system and requests to view the family tree, the server sends the virtual display data that meets the permissions to the family member according to the corresponding permissions.

Each family member can also create a family tree account, and specify the family tree where it is located when creating the family tree account. After the server approves the creation request, it will open the corresponding data and information to it according to the authority specified by the administrator user. resource.

In order to improve the utilization rate of family tree data, people outside the family tree can register with the system as application type users. For example, managers of the hospital system, experts in a certain field, scholars, etc. Therefore, when the user starts the client and chooses to create an account, the family tree system of the present invention can have multiple options, such as the user in Figure 7, since he is the creator of a family tree, he chooses "family tree user" , similarly, family members should also select "Family Tree User" when creating an account. When the user currently creating an account wants to use the family tree data in the system, he can choose "application user", which means that the user can query the data of a certain aspect of the family tree data according to the type of application provided by the system.

Various types of users can send corresponding requests through the client in the terminal device, and the server provides them with corresponding virtual display data according to the user type and request content.

FIG. 8 is a flowchart of a process for a user to view family tree data according to an embodiment of the present invention. In this embodiment, the terminal device used by the user may be AR glasses, a smart phone, an HMD, etc., in which a client of the family genealogy system is installed, and the terminal device can perform data transmission with the server through the network. This embodiment includes the following steps:

In step S21a, the user starts and logs in the family genealogy system client in the terminal device.

In step S21b, the terminal device provides a client interface in response to the user's operation on the client of the family tree system.

In step S22a, the user sets viewing conditions and inputs viewing instructions.

In step S22b, the terminal device receives the input viewing condition data, and generates a viewing request, which includes the user's account information, viewing condition data, and terminal device information.

In step S23b, the terminal device sends a viewing request to the server of the family tree system.

Step S21c, after receiving the viewing request, the server parses out the user's account information, viewing condition data and terminal device information, for example, the current user is using a smart phone. And according to the user's account information to determine the category and authority of the user classification.

In step S22c, the server collects data from corresponding nodes according to viewing conditions and permissions.

In step S23c, the server extracts virtual data such as corresponding virtual scenes, virtual models, or personal information labels from the virtual display data according to the collected data.

In step S24c, the server determines a display mode based on the user terminal equipment information, and fuses the extracted virtual data according to the display mode to obtain virtual display data.

In step S25c, the server sends the virtual display data to the terminal device.

In step S24b, the terminal device receives the data returned by the server to display the data.

Wherein, when the terminal device is a smart phone, the returned data can be displayed in the form of AR or MR. For example, when the user is a family tree user and the viewing condition entered is a time period, the returned data is a virtual scene of all events of all family members meeting the time period. Taking the MR display method as an example, the terminal device creates a digital space that overlaps with the current real scene based on the current user's environment image. When receiving the returned family tree virtual scene data, according to the number of virtual scenes and their relationships to be integrated at the same time, Identify the corresponding number of spaces in the current digital space, integrate the virtual scene into the corresponding space according to the time sequence, and render the current space according to the needs. Since the fusion is performed according to the time sequence during the fusion, the user starts from the current intelligence. Multiple consecutive virtual scenes can be viewed sequentially and dynamically on the mobile phone.

Take the tree diagram of a family tree in Figure 4B as an example. There are currently 11 family members in total. The large image in the picture is the blood-related member of the family, and the small image is the affinity relationship with the family member. a member of. When the user displays the transition process of the birthplace of this section of the family tree through the smartphone in the living room, the smart phone constructs a three-dimensional space overlapping with the living room, and displays it in one part of the three-dimensional space of the living room, such as a corner on the sofa. The birth scene of the earliest first-generation member currently born, such as the birth scene of member FB1 first, then the birth scene of member F11, then the birth scene of the second-generation members F21, F22, and F23, and then the birth scene of the third-generation member , and finally the birth scene of the members of the fourth generation. The style of buildings and houses in each birth scene matches its era, and each birth scene corresponds to geographically related elements, such as mountains, seaside, cities, etc., and each birth scene is also optionally equipped with text descriptions , voice instructions, sound effects, etc., so that users can enjoy immersive visual and auditory effects when watching. For each scene, the user can also interact with it, for example, click and enlarge the description text in a certain scene, choose to play and stop the sound effect, interact with the character model in the scene, and so on.

When the user is an application type user, he can set various keywords in the query condition to query certain application data. For example, when the user wants to view the family medical history of a family, he can input "family disease" in the client interface , the server will return the virtual display data about the composition of the family disease scene, such as the typical scene of each generation of family members getting sick and seeking medical treatment for their family disease, supplemented by text or voice instructions. Anthropology experts can enter keywords such as "climate", "topography", "region", and "life span" in the client interface to query specific data in these aspects from multiple family genealogy data. Anthropology experts can use these specific data to analyze the relationship between human health indicators, intellectual development, etc. and these factors.

It should be noted that, for the sake of clarity, each embodiment of the present invention is expressed as a series of actions or combinations of processes. Those skilled in the art should know that the implementation process is not limited by the sequence of actions or processing described, and some steps in the embodiments of the present invention may be processed in other sequences or at the same time.

On the other hand, as shown in FIG. 9, FIG. 9 is a functional block diagram of a family tree system according to an embodiment of the present invention, the system includes a data entry module 1, a data association module 2, a virtual scene creation module 3, a first application The processing module 4 and the display module 5, the data entry module 1 obtains the personal information of natural persons and the family relationship among family members based on the entry interface or interface. The data association module 2 is connected with the data entry module 1, establishes the family tree of the natural person based on the family relationship of the natural person, and establishes the association relationship between the family tree and the personal information of each family member; the virtual scene creation module 3 is based on Family genealogy data and family member personal information create family tree virtual display data, the virtual display data includes one or more virtual scenes, virtual models and personal information labels of one or more family members; the first application processing module 4 Extract the corresponding type of first application data from the family tree data based on the application type, extract the virtual display data from the family tree virtual display data based on the first application data, and generate the first virtual display in the form of AR, VR or MR Show data. The display module 5 is connected to the first application processing module 4, and based on a user request corresponding to an application type, displays the first virtual display data in an AR, VR or MR manner.

In terms of implementation, the family genealogy system includes a server and a client, wherein the data entry module 1 can be realized by the client, and the data association module 2, the virtual scene creation module 3 and the first application processing module 4 can be realized by the server, The presentation module 5 can be implemented jointly by the server and the client. The client is located in the user terminal device. In addition to realizing the data entry module 1 and the display module 5 in cooperation with the server, it also includes other modules, such as an interface module that provides various interfaces, and is used to realize some functions of data entry processing. Processing modules, such as image processing, semantic recognition, speech recognition and other modules. In order to realize the display of virtual data in the form of AR, VR or MR, the client also includes matching processing modules, such as V-SLAM module, rendering module, positioning module and so on. Those skilled in the art can implement these functional modules with reference to the prior art. I won't repeat them here. The server includes one or more servers, databases, and the like.

In another embodiment, as shown in FIG. 10, FIG. 10 is a functional block diagram of a family tree system according to another embodiment of the present invention. In addition to the modules in FIG. 9, the system also includes a plurality of second application processing Modules, such as the query processing module 81, the prediction processing module 82 and the statistical module 8n, etc. in the figure, wherein the query processing module 81 performs query in a family genealogy data or all family genealogy data based on the user's query request To meet the user's query requirements, the query processing module 81 can be implemented using any search engine. For example, it is possible to query the personal information of their relatives for the user, including current address, contact information and other information. The forecasting processing module 82 may have various forecasting functions, in which various trained forecasting models are stored, and complete the forecasting of a certain content according to the user's needs. For example, predict the appearance of the child in a few years according to the input photo of the child; another example, according to the identity information of the two natural persons input, by querying the family disease data and gene data of their family tree, it is predicted that the children of the two will suffer from family diseases after they get married. Or the probability of a certain disease; another example is based on the identity information of two natural persons input, by querying their family genealogy data to determine the blood relationship between the two, so as to predict whether the two are suitable for marriage in terms of blood relationship, etc. Various statistical indicators are stored in the statistical module, and data statistics are performed according to one or more specific statistical indicators selected by the user. Statistical data such as genetic genes, family diseases, place of birth, life expectancy, occupation, education, etc. , and provide corresponding constraints, or perform statistics according to the constraints given by the user, the above data processed by different second application processing modules are collectively referred to as the second application data. After the second application processing module obtains the second application data, based on the second application data, it extracts the corresponding virtual data from the family tree virtual display data, and generates the second virtual display data displayed in AR, VR or MR mode, and the The display module 6 is provided to the user.

In another embodiment, as shown in FIG. 11, FIG. 11 is a functional block diagram of a family tree system according to another embodiment of the present invention. The system also includes a system interface 7 on the basis of FIG. 10, which is connected with the data entry module 1 is connected with the display module 5 for communicating with the third-party system. The third-party system such as hospital system, public security system, education system or civil affairs bureau system, etc., through the system interface 7, the data entry module 1 in the family genealogy system in this embodiment can be obtained from other third-party systems Data required for database download. When the third-party system needs to obtain certain application data, it sends its request to the display module 5 through the system interface 7, and when the display module 5 receives the request, it sends the request to the first application processing module 4 or other first application processing module 4. 2. Apply the processing module for processing, and then return the processed virtual display data to the third-party system through the system interface 7.

Fig. 12 is a functional block diagram of the C-S architecture of the family genealogy system according to an embodiment of the present invention. In this embodiment, the user terminal includes a client, which communicates with the server through a communication module. The server includes a user management module and a data processing module. The data processing module includes, for example, various modules in FIGS. 9-11, such as a data association module 2, a virtual scene creation module 3, a first application processing module Query module 81, prediction processing module 82, statistics module 8n and various application processing modules not listed in FIG. 11 . The user management module classifies users and sets permissions. The server also includes various databases, such as family tree database, resource database and user database in the figure. The family tree database stores personal files of family members in each family tree, tree diagram data of the family tree, corresponding virtual display data and various application data. The resource database is used to store various resource data or resource links required by the system, such as map data, 3D models, various audio, sound effects, typical virtual scenes, and so on. The user database stores user information of registered accounts.

Those skilled in the art should understand that the embodiments described herein are preferred embodiments, and the actions, steps, modules or units involved are not necessarily required by the embodiments of the present invention. In the foregoing embodiments, the descriptions of the embodiments of the present invention have different emphases on each embodiment, and for the parts not described in detail in a certain embodiment, refer to the relevant descriptions of other embodiments.

13 is a schematic structural diagram of an electronic device 60 provided by an embodiment of the present invention. The electronic device 60 includes a processor 61, a memory 62, and a communication bus for connecting the processor 61 and the memory 62, wherein the memory 62 stores information that can A computer program running on the processor 61, when the processor 61 runs the computer program, it can execute or realize the steps in the methods of the various embodiments of the present invention. Electronic device 60 also includes a communication interface for receiving and sending data. The electronic device 60 may be the server in the embodiment of the present invention, or various terminal devices in the embodiment of the present invention. Electronic devices may also be referred to as computing devices where appropriate.

In some embodiments, the processor 61 may be a central processing unit (Central Processing Unit, CPU), a graphics processing unit (graphics processing unit, GPU), an application processor (application processor, AP), a modem processor, an image Signal processor (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, neural network processor (neural-network processing unit, NPU), etc.; The processor 61 can also be other general-purpose processors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices , discrete hardware components, etc. A general-purpose processor may be a microprocessor, any conventional processor, or the like. Among them, the neural network processor NPU can quickly process input information by referring to the biological neural network structure, and can also continuously carry out self-learning. Applications such as intelligent cognition, such as image recognition, face recognition, semantic recognition, speech recognition, text understanding, etc., can be realized through the NPU electronic device 60 .

In some embodiments, the memory 62 can be an internal storage unit of the electronic device 60, such as a hard disk or memory of the electronic device 60; the memory 62 can also be an external storage device of the electronic device 60, such as a plug-in Hard disk, smart media card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card, flash memory card (Flash Card), etc. The memory 62 may also include both an internal storage unit of the electronic device 60 and an external storage device. The memory 62 can be used to store operating system, application program, boot loader (BootLoader), data and other programs, such as program codes of computer programs. Memory 62 includes, but is not limited to, random access memory (random access memory, RAM), read-only memory (read-only memory, ROM), erasable programmable read-only memory (erasable programmable read only memory, EPROM) or portable Read-only memory (compact disc read-only memory, CD-ROM). The memory 62 is used to store program codes executed by the electronic device 60 and transmitted data. The memory 62 can also be used to temporarily store data that has been output or will be output.

Those skilled in the art can understand that FIG. 13 is only an example of the electronic device 60, and does not constitute a limitation to the electronic device 60. The electronic device 60 may include more or fewer components than shown in the figure, or combine certain components, or It includes different components, for example, it may also include input and output devices, network access devices, and so on.

Fig. 14 is a schematic diagram of a software structure of a terminal device according to an embodiment of the present invention. Taking the mobile phone operating system as the Android system as an example, in some embodiments, the Android system is divided into four layers, which are respectively: application program layer, application program framework layer (framework, FWK), system layer and hardware abstraction layer, layer and Layers communicate through software interfaces.

First of all, the application layer can include multiple application packages, and the application packages can be various application apps such as call, camera, video, navigation, weather, instant messaging, education, etc., or applications based on AR technology .

Second, the application program framework layer FWK provides an application programming interface (application programming interface, API) and a programming framework for applications in the application program layer. The application framework layer may include some predefined functions, such as functions for receiving events sent by the application framework layer.

The application framework layer may include window managers, resource managers, and notification managers, among others.

Among them, the window manager is used to manage window programs. The window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. Content providers are used to store and retrieve data and make it accessible to applications. Said data may include video, images, audio, calls made and received, browsing history and bookmarks, phonebook, etc.

Among them, the resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files and so on.

Among them, the notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify the download completion, message reminder, etc. The notification manager can also be a notification that appears on the top status bar of the system in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, prompting text information in the status bar, issuing a prompt sound, vibrating the electronic device, and flashing the indicator light, etc.

In addition, the application framework layer may also include a view system, and the view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and the like. The view system can be used to build applications. The display interface may be composed of one or more views, for example, the display interface of the SMS notification icon may include a view for displaying text and a view for displaying pictures.

Third, the system layer may include multiple functional modules, such as a sensor service module, a physical state recognition module, a three-dimensional graphics processing library (eg, OpenGLES), and so on.

Among them, the sensor service module is used to monitor the sensor data uploaded by various sensors at the hardware layer to determine the physical state of the mobile phone; the physical state recognition module is used to analyze and recognize user gestures, faces, etc.; the 3D graphics processing library is used to Realize 3D graphic drawing, image rendering, compositing, and layer processing, etc.

In addition, the system layer can also include surface managers and media libraries. The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of various commonly used audio and video formats, as well as still image files, etc.

Finally, the Hardware Abstraction Layer is the layer between hardware and software. The hardware abstraction layer may include a display driver, a camera driver, a sensor driver, etc., and is used to drive related hardware of the hardware layer, such as a display screen, a camera, and a sensor.

Embodiments of the present invention also provide a computer-readable storage medium, which stores computer programs or instructions, and when the computer programs or instructions are executed, the steps in the methods involved in the above-mentioned embodiments are implemented.

An embodiment of the present invention also provides a computer program product, including a computer program or an instruction. When the computer program or instruction is executed, the steps in the methods involved in the above-mentioned embodiments are implemented. Exemplarily, the computer program product may be a software installation package.

Those skilled in the art should know that the methods, steps or functions of related modules/units described in the embodiments of the present invention may be realized in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it may be implemented in whole or in part in the form of a computer program product, or may be implemented in a manner in which a processor executes computer program instructions. Wherein, the computer program product includes at least one computer program instruction, and the computer program instruction can be composed of corresponding software modules, and the software modules can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, mobile hard disk, CD-ROM (CD-ROM) or any other form of storage medium known in the art. The computer program instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available medium may be a magnetic medium (for example, a floppy disk, a hard disk, or a magnetic tape), an optical medium, or a semiconductor medium (such as an SSD).

Regarding the devices/products described in the above embodiments, the modules/units contained therein may be software modules/units, or hardware modules/units, or partly software modules/units, partly hardware modules/units. For example, for an application or a device/product integrated in a chip, each module/unit contained in it may be implemented by hardware such as a circuit, or at least some modules/units may be implemented by a software program, running on the chip integrated in the chip. The processor, and the rest of the modules/units are realized by means of hardware such as circuits. As another example, for an application or a device/product integrated in a terminal, each module/unit contained in it may be realized by means of hardware such as a circuit, or at least some modules/units may be realized by means of a software program and run in the integrated terminal inside the terminal. processor, and the rest of the modules/units can be realized by means of hardware such as circuits.

The above is only a specific implementation of the present invention, and those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the above-described systems, modules and units can refer to the foregoing method embodiments The corresponding process in , will not be repeated here. It should be understood that the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present invention, and these modifications or replacements should cover all Within the protection scope of the present invention.

Claims (15)

1. A family tree system processing method, characterized in that, comprising: Obtain the personal information of natural persons and the family relationship among family members based on the input interface or interface; Establish a family tree of a natural person based on the family relationship of the natural person, and establish the relationship between the family tree and the personal information of each family member; Create family tree virtual display data based on family genealogy data and family member personal information, said virtual display data including one or more of one or more virtual scenes, virtual models and personal information labels of one or more family members; Based on the application type, extract the corresponding type of first application data from the family tree data, and based on the first application data, extract one or more of the corresponding virtual scene, virtual model and personal information label from the family tree virtual display data or, and fused into the first virtual display data displayed in AR, VR or MR; and Based on a user request corresponding to an application type, the first virtual display data is displayed in an AR, VR or MR manner. 2. The method according to claim 1, wherein the personal information of the natural person includes a plurality of attribute information, and the attribute is one of general attributes, physiological attributes, geographical attributes, social attributes, communication attributes or many. 3. The method according to claim 2, wherein the attribute information is expressed in one or more of text, image, audio and video. 4. The method of claim 1, further comprising: The first application data is processed according to the application requirement information to generate the second application data meeting the application requirement. 5. The method according to claim 4, further comprising: The second virtual display data displayed in AR, VR or MR manner is generated based on the second application data. 6. The method of claim 5, further comprising: Determine the application type and/or application requirement information corresponding to the third-party system based on the third-party system request; and The corresponding first virtual display data and/or second virtual display data are obtained based on the application type and/or application requirement information, and provided to a third-party system through a system interface. 7. The method according to claim 1, further comprising: During the process of creating a user account for the user, the user type and authority are determined for the user. 8. The method according to claim 1, further comprising: When a user request is received, extract the user terminal device information from the user request; and Based on the content requested by the user and the information of the user terminal device, it is determined to display the first virtual display data in an AR, VR or MR manner. 9. A family genealogy system, characterized in that it comprises: A data entry module configured to obtain personal information of natural persons and family relationships among family members based on entry interfaces or interfaces; A data association module, which is connected to the data entry module, configured to establish a family tree of a natural person based on the family relationship of the natural person, and establish an association relationship between the family tree and the personal information of each family member; The virtual scene creation module is configured to create virtual display data of the family tree based on the family tree data and personal information of family members, and the virtual display data includes one or more virtual scenes, virtual models and personal information tags of one or more family members one or more of The first application processing module is configured to extract a corresponding type of first application data from the family tree data based on the application type, and extract a virtual scene, a virtual model, and a personal information label from the family tree virtual display data based on the first application data One or more of them, and merged into the first virtual display data displayed in the form of AR, VR or MR; and A display module, which is connected to the first application processing module, is configured to display the first virtual display data in an AR, VR or MR manner based on a user request corresponding to an application type. 10. The system according to claim 9, further comprising a plurality of second application processing modules, which are respectively connected to the first application processing module, configured to process the first application according to the application requirement information The data is processed to generate second application data that meets the application requirements; second virtual display data displayed in AR, VR or MR is generated based on the second application data. 11. The system according to claim 9, further comprising a system interface connected to the presentation module, configured to receive a third-party system request, based on the application type and/or The corresponding first virtual display data and/or second virtual display data are obtained from the application requirement information, and provided to a third-party system through the system interface. 12. The system according to claim 9, further comprising a user management module configured to determine a user type and permissions for the user during the process of creating a user account for the user. 13. A server, characterized in that it includes a processor and a memory that stores computer program instructions; when the processor executes the computer program instructions, it realizes the family tree system according to any one of claims 1-8 Approach. 14. A computer-readable storage medium, wherein computer program instructions are stored on the computer storage medium, and when the computer program instructions are executed by a processor, the method according to any one of claims 1-8 is implemented. Family genealogy system processing method. 15. A computer program product, characterized in that it includes computer program instructions, and when the computer program instructions are executed by a processor, the family tree system processing method according to any one of claims 1-8 is implemented.

Images