WO2018099400A1 - 基于增强现实的家居设计系统和方法 - Google Patents

基于增强现实的家居设计系统和方法 Download PDF

Info

Publication number
WO2018099400A1
WO2018099400A1 PCT/CN2017/113605 CN2017113605W WO2018099400A1 WO 2018099400 A1 WO2018099400 A1 WO 2018099400A1 CN 2017113605 W CN2017113605 W CN 2017113605W WO 2018099400 A1 WO2018099400 A1 WO 2018099400A1
Authority
WO
WIPO (PCT)
Prior art keywords
tool
virtual
scene
model
virtual object
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2017/113605
Other languages
English (en)
French (fr)
Inventor
高振东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Positec Power Tools Suzhou Co Ltd
Original Assignee
Positec Power Tools Suzhou Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Positec Power Tools Suzhou Co Ltd filed Critical Positec Power Tools Suzhou Co Ltd
Publication of WO2018099400A1 publication Critical patent/WO2018099400A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]

Definitions

  • the present invention relates to the field of augmented reality technology, and in particular, to an augmented reality based home design system.
  • the invention also relates to an augmented reality based home design method.
  • Augmented Reality (AR) and Virtual Reality (VR) have been applied in the field of home design.
  • some home improvement stores have set up virtual reality technology experience zones. After wearing VR glasses/helmets, consumers can see virtual three-dimensional space and experience the products in the virtual three-dimensional space.
  • the above solution can only experience the scene defined by the merchant in the merchant's store; the virtual space, tools, commodities and the actual home environment of the consumer experience trial, the actual design and modification requirements are completely different, and cannot be directly presented. Consumers' own home design and transformation effects.
  • an augmented reality based home design system which includes:
  • An environmental information collector for detecting and collecting three-dimensional information of the environment
  • a modeling module including a scene simulator and an object simulator, respectively generating a virtual scene model and a virtual object model; the scene simulator generating the virtual scene model based on the three-dimensional information collected by the environment information collector; the virtual object model having a spatial attribute and a morphological attribute, the spatial attribute including a spatial position and/or an angle of the simulated object, the morphological attribute including a self-morphology of the simulated object and/or a connection relationship with other objects or scenes;
  • a display module based on the positioning of the virtual object model in the virtual scene model, superimposing the virtual object model on the real scene, or superimposing the virtual scene model and the virtual object model on the real scene;
  • the input module is configured for the user to manipulate the virtual object model or the virtual tool model to change a spatial attribute and a morphological attribute of the simulated object.
  • the object simulator generates the virtual object model based on the three-dimensional information collected by the environment information.
  • the home design system stores, or is connected to, a database, and the object simulation The device retrieves information from the database and generates a virtual object model.
  • the morphological attribute includes at least one of a shape attribute, a size attribute, a connection attribute, and an assembly attribute.
  • the home design system further includes a tool modeler, and the tool modeler generates a virtual tool model, which can simulate the processing effect of the real tool and correspondingly change the shape attribute of the simulated object.
  • the tool modeler obtains attribute data of the virtual tool model from an internal tool library or an external data source, the attribute data includes type data of the tool and working parameter data, and the tool simulation module simulates according to the attribute data.
  • the processing effect of realistic tools is not limited to
  • the external data source is a tool itself or a cloud tool database.
  • the environmental information collector is one of a depth camera, a structured light detector, and a laser radar.
  • a purchasing module is further included, and the purchasing module generates a service related to the simulation object and or a commodity purchase suggestion.
  • the purchasing module generates a purchase suggestion of a tool or a service required to process the real object based on a morphological attribute of a simulated object selected by the user.
  • the present invention also provides a combination of an augmented reality based home design system and a tool, comprising the home design system of any of the preceding clauses, further comprising a tool, the home design system enters attribute data of the tool, and is based on The attribute data of the tool changes the morphological properties of the virtual object model.
  • the home design system includes a tool information reader to identify the tool and enter attribute data of the tool.
  • the home design system generates a tool use guiding step according to the user's manipulation of the simulated object, and displays it to the user.
  • the tool is a power tool.
  • the present invention also provides a tool having tool property indicia for reading by the aforementioned home design system to obtain attribute data for the tool.
  • tool attributes are marked as a mark pattern, an RFID tag or a built-in memory.
  • the tool is a power tool.
  • the invention also provides a home improvement design method based on augmented reality, comprising the following steps:
  • Detecting and collecting scene data of a real scene where the scene data includes image information and depth information;
  • a user-manipulatable virtual object model having spatial and morphological attributes, the spatial attributes including spatial locations and/or angles of the simulated objects, the morphological attributes including the virtual object model's own form and/or Connection relationship with other objects or scenes;
  • the invention also provides a home improvement design method based on augmented reality, comprising the following steps:
  • a user-manipulatable virtual object model having spatial and morphological attributes, the spatial attributes including spatial locations and/or angles of the simulated objects, the morphological attributes including the virtual object model's own form and/or Connection relationship with other objects or scenes;
  • Service/commodity purchase suggestions are provided based on user manipulation of the virtual object model.
  • the above-mentioned augmented reality-based home design system collects three-dimensional information in the user's home by using the environmental information collector, and based on the three-dimensional information modeling, improves the pertinence of the augmented reality home design system to the specific environment and needs of the user.
  • a tool simulator real-time simulation of the real tool's processing effect on home design modification work, increasing the fun of home DIY work.
  • FIG. 1 is a block diagram of a home design system of an embodiment
  • an augmented reality based home design system of an embodiment includes a controller 1 including a scene simulator 3, an object simulator 4, a tool simulator 5, an environment information collector 7, and a display module 8. And the input module 9. Communication between the home design system and the cloud server 6.
  • the environmental information collector 5 is capable of scanning three-dimensional information of the environment and transmitting it to the controller.
  • the environment information collector scans the real scene in the environment, and transmits the three-dimensional information of the real scene to the scene simulator 3, and the scene simulator 3 generates a virtual scene model based on the received information.
  • the environment information collector can also scan the real object in the environment, send the three-dimensional information of the real object to the object simulator, and the object simulator 4 generates the virtual object model based on the received information.
  • the environmental information collector can also scan the real-world tools in the environment, send the information of the real tools to the object simulator, and the object simulator 4 generates a virtual object model based on the received information.
  • the scene constitutes the background environment in which the object is located
  • the object is the target object to be processed and changed in the scene
  • the tool is an instrument for processing and changing the target object.
  • the scene is the living room in the user's home
  • the object is a photo frame that needs to be hung on the wall
  • the tool is an electric drill.
  • the object simulator and tool simulator can also generate virtual object models and virtual tool models based on other sources, such as information sent by the server. Specifically, it will be described later.
  • the environmental information collector can also collect image information in the environment.
  • the virtual scene model and the virtual object model generated based on the aforementioned information are a fusion model of three-dimensional information and image information.
  • the environmental information collector is a multi-view depth camera
  • the multi-view depth camera includes at least two cameras that utilize the coordinate difference of the target point imaged on the two views taken by the two cameras, and the target The inverse relationship between the distances from the point of the imaging plane to obtain the distance information of the target point.
  • the environment is first scanned by a multi-depth camera, and then the corresponding modeling module (such as scene modeler, object modeler, etc.) in the controller is used to decompose the photo into pictures and corresponding Depth information and record it. Then, through the built-in 3D reconstruction algorithm, the modeling module splicing the image into a panoramic image, and then restoring the recorded depth information to the image, and finally obtaining a three-dimensional model of the scanned object.
  • the corresponding modeling module such as scene modeler, object modeler, etc.
  • the environmental information collector is a structured light scanner.
  • the structured light is a projected light in a known spatial direction, which can be a point, a line, or a grid projected by various lasers, projectors, or other light sources. Grid line.
  • the structured light scanner can include a laser emitter and a camera.
  • the laser emitter projects a point laser at an angle, the position of the reflection point in the X-axis of the camera, and the linear distance of the corresponding reflection point from the measuring device. If a linear laser is projected, a 3D point cloud can be obtained after 360° scanning by a structured light scanner. In one embodiment, there may be multiple cameras.
  • the structured light scanner includes a camera to capture information such as light and dark, color, and the like of the object being scanned. Similar to the solution of the multi-view depth camera, after the structured light scanner obtains the environmental three-dimensional information, the corresponding modeling module in the controller records the image and the corresponding depth information. Then, through the built-in 3D reconstruction algorithm, the modeling module splicing the image into a panoramic image, and then restoring the recorded depth information to the image, and finally obtaining a three-dimensional model of the scanned object.
  • the environmental information collector is a range finder, such as a laser range finder.
  • the range finder rotates 360° and scans the surrounding objects to obtain three-dimensional information of the environment and sends it to the controller 1.
  • the modeling module in the controller 1 creates a virtual three-dimensional model based on the received information, such as a virtual scene model and a virtual object model. , virtual tool models, etc.
  • the controller 1 includes a scene simulator, an object simulator, and a tool simulator. It should be noted that each of the above simulators is an unstructured functional module.
  • the entity of the controller 1 includes conventional components such as a microprocessor, a volatile memory, a non-volatile memory, and the like. Model-related algorithms are stored in the non-volatile memory.
  • the scene simulator is used to receive scene information to create or update a virtual scene model.
  • the scene information is derived from scene three-dimensional information and image information collected by the environmental information collector, such as information of a living room, a study room, and the like.
  • the scene information may also come from a server in the cloud, or a user's input or the like.
  • the virtual scene model includes three-dimensional spatial information and surface image information of the scene; in some embodiments, material information of the object in the scene, and the like.
  • the object simulator is used to receive object information to build a virtual object model.
  • the object information is derived from three-dimensional information and image information of objects collected by the environmental information collector.
  • the object information may also come from a server in the cloud or a user input or the like.
  • the object to be processed or modified may not exist in the user's home, but may be selected by the user in an object library, modeled by the object simulator and projected in the virtual scene model, or a realistic scene. in.
  • a user wants to experiment with adding a coffee table in their living room, they can Select a coffee table in the local or cloud object library, and then display it in the virtual scene model or the real scene by the object simulator, so that the user can observe whether the size of the coffee table is appropriate, whether the style matches the existing layout, and the like.
  • the user then adjusts parameters of the virtual coffee table model through the input module, such as spatial position, color, etc., and the display module adjusts the display content according to the user's input to reflect the user's operation.
  • the display module adjusts the display content in real time according to the input of the user.
  • the object simulator can create some virtual object models based on the information of the environment information collector, and also select models from the object library. Still taking the living room as an example, the system scans the scene information by the environment information collector, and the scene simulator creates a virtual scene model of the living room; scans the object information such as the sofa and the floor lamp, and the object simulator creates a virtual sofa model and a virtual floor lamp model, and then The user then selects a coffee table from the object library for the object simulator to create a virtual coffee table model.
  • the display module fuses the virtual scene model and each virtual object model, and displays the placement effect of the coffee table to the user through the display module, and through the input module, the user adjusts the parameters of each virtual object model and adjusts the effect. Displayed again through the display module.
  • the user can observe objects that do not exist in the real scene, such as the placement effect of the coffee table in the scene, and can also adjust the objects existing in the real scene, such as sofas, floor lamps, etc., thereby After observing and testing the addition of household items, how do existing household items need to be moved, whether the space is sufficient after moving, and whether the effect is satisfactory.
  • the object model established by the object simulator includes shape-related attribute parameters such as spatial attributes and color attributes, as well as morphological attributes. Morphological properties are related to the modifiable nature of the object. For example, morphological attributes include the object's decomposable attributes, such as whether it can be broken into different blocks; or the object's machinable attributes, whether any position or specific position can be cut or polished; or the object's installation properties, such as whether it can and other objects Or the scene is mounted or spliced in some way, such as bonding, welding, inlaying, threading, and the like.
  • the tool simulator is used to build a virtual tool model.
  • Both the virtual scene model and the virtual object model described above have three-dimensional size information corresponding to the real thing.
  • the virtual tool model may be different, may not have three-dimensional size information, but has type information and processing/modification capability information, for example.
  • the type information is that the virtual tool model is an electric drill, or a reciprocating saw, a sanding machine, etc.; the processing/modification capability information is the power, working time, processing aperture of the electric drill, etc.; or the power of the reciprocating saw, the workpiece that can be cut Type and so on.
  • the virtual tool model can also have three-dimensional size information.
  • the tool simulator builds a virtual tool model based on tool data obtained from a database of a server or local storage; in another embodiment, the tool simulator builds a virtual tool model based on a real tool, eg, reality
  • the tool has a readable mark such as a two-dimensional code, a magnetic mark or an RFID tag, wherein the tool information is recorded, and the home design system has a scanner, a magnetic sensor or an RFID reader to read the tool information.
  • the tool simulator builds a virtual tool model based on the tool information that is read.
  • the foregoing readable tool information includes only simple tool type information, and the tool simulator reads detailed tool information from the database according to the tool type information, such as the processing/modification capability information described above.
  • the display module is a transparent or translucent display.
  • the advantage of this type of display is that the user can see the real environment through the display, so that the virtual model and other required information are superimposed and displayed in the real environment.
  • the display module can be a helmet or eye with a transparent display lens.
  • the display module has a built-in position and attitude sensor, which can sense the change of its own position and posture and the relative position relationship with the user, so as to ensure that the displayed content changes with the user's perspective and is always in the correct position. For example, when the position and posture of the user change, the virtual scene model and the real scene are always coincided, and the virtual object model is in the virtual scene model, that is, the set position in the real scene.
  • the display module can be a handheld display, a projector, or the like.
  • the input module may be a touch panel, and the user inputs through a touch operation.
  • the input module may be a natural interactive interface device, such as a gesture sensing device, which senses the natural gesture of the user for corresponding control, such as dragging, slamming, etc. of the user.
  • the input device and the display module are fused together, for example, the user controls the virtual objects and virtual tools displayed by the display device in the virtual space by natural gestures, and executes various instructions of the user. For example, the user can directly drag the virtual coffee table in the virtual space, and it will move to the dragged position accordingly, and the user can also use the virtual electric drill to hang the virtual photo frame on a certain wall.
  • the home design system simulates the processing effect of the real tool on the real object.
  • each simulator of the home design system first completed the modeling work, for example, establishing a virtual scene model, a virtual tool model, and a virtual object model.
  • the display module is in the user's field of view.
  • the user operates the virtual tool model through the aforementioned operation module, and the display module displays the processing process in real time as the user operates, and displays the processing result after the processing is completed.
  • the user can manipulate a virtual jig saw to cut a circular plate on a virtual wooden board.
  • the virtual object model is superimposed and displayed on the real object.
  • the processing process is superimposed and displayed on the real object in real time.
  • the properties of the virtual object model are changed, for example, the shape property of the virtual wood board is changed from a square to a circle.
  • the home design system provides merchandise/service provisioning advice to the user.
  • the home design system has a built-in local database or communicates with the cloud database to obtain product/service provisioning information.
  • the product/service supply information includes, but is not limited to, the purchase or rental information of the real object corresponding to the virtual object model, such as the purchase link or purchase address, the selling price or the rent of the sofa, the photo frame, etc.; the realistic tool corresponding to the virtual tool model
  • Purchase or rental information such as electric drills, reciprocating saws, screwdrivers, and other purchase links or purchase addresses, prices, or rent.
  • the home design system when the user enters the home design system and performs home design, if the user selects a home item and selects a virtual object model from the database, the home design system simultaneously acquires the purchase of the item corresponding to the virtual object model from the database. Linking and displaying to the user through the display model, the user can change the position of the virtual object model through the input module, switch the virtual object model of other models, etc., if the effect is satisfactory, the purchase can be made through the product/service supply information, for example, Direct ordering and payment in the home design system, delivered by the merchant. The user can also purchase or rent the real tools corresponding to the selected virtual tool model when designing the home through the home design system.
  • the specific process is similar to the above item purchase process, and will not be described again.
  • Users can also purchase or obtain home design/modification services when designing their home through a home design system. For example, after the user selects the goods to be purchased through the home design system, and determines the decoration and modification scheme, the home design system directly packages the overall design solution to generate a purchase link. After the user confirms the purchase, the merchant provides the goods to the door. And furnishings, modification services. Specifically, for example, in the home design system, the user confirms that a set of a specific type of sofa is placed in the living room by the augmented reality technology, a plurality of paintings are hung on the position set on the wall, and the carpet is laid. Then, the overall design plan will be uploaded to the merchant, and the merchant will send the sofa, painting and carpet to the customer's home according to the overall design scheme. In the middle, and install the design plan, the carpet is laid, the sofa is placed in place, and the painting is mounted on the wall.
  • the home design system provides the user with design/modification operational guidance recommendations.
  • the home design system can provide step-by-step instructions for the user to disassemble the operation instructions. For example, the user needs to make a bookcase by himself. After the design is determined and the raw materials are prepared, the home design system guides the user through the display module.
  • the display module can be used as a projector or with a transparent or translucent display. The glasses or helmet of the instrument, the display module directly superimposes the operation instructions on the real scene, objects and tools.
  • bookcase production requires sheet cutting, sheet polishing, surface painting, mounting hole fabrication, splicing and fixing, etc.; tools such as jigs, cutting saws, sanders, brushes or painting equipment, and electric drills are required. Accordingly, the home design system is broken down into multiple operational steps and displays the steps currently in progress on the display module, the materials, tools, and how the user is required to operate.
  • the first step is to clamp the plate on the fixture, then the display module marks the position of the plate and the fixture, and tells the user to clip the plate to the fixture through animation or text;
  • the second step is to use
  • the cutting saw cuts the fixed length of the sheet, and after the user completes the first step, the display module cancels the prompt in the first step, and marks the cutting saw, and displays a position on the sheet to be cut on the fixture.
  • the guidance of the home design system is similar, and will not be described again.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Evolutionary Computation (AREA)
  • Geometry (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本发明涉及一种基于增强现实的家居设计系统，包括：环境信息收集器，；建模模块，包括场景模拟器、对象模拟器；场景模拟器基于环境信息收集器收集的三维信息生成虚拟场景模型；显示模块，在现实场景上叠加显示所述虚拟对象模型、或在现实场景上叠加显示虚拟场景模型及虚拟对象模型；输入模块，供用户操控以改变所述模拟对象的空间属性和\或形态属性。上述基于增强现实的家居设计系统，通过使用环境信息收集器收集用户家中的三维信息并基于该三维信息建模，提高了增强现实家居设计系统对用户具体环境和需求的针对性。

Description

基于增强现实的家居设计系统和方法 技术领域
本发明涉及增强现实技术领域，尤其涉及一种基于增强现实的家居设计系统。
本发明还涉及一种基于增强现实的家居设计方法。
背景技术
增强现实技术(AR)和虚拟现实技术(VR)在家居设计领域已经得到应用。例如，部分家装卖场设立了虚拟现实技术体验区，消费者在佩戴VR眼镜/头盔后，能够看到虚拟的三维空间，并在虚拟三维空间中体验卖场的商品。然而，上述方案只能在商家的卖场中，体验商家所限定的场景；消费者所体验试用的虚拟空间、工具、商品和消费者的实际家居环境，实际设计和改装需求截然不同，不能直接呈现消费者自身的家居设计和改造效果。
发明内容
基于此，本发明提供了一种基于增强现实的家居设计系统，其包括：
环境信息收集器，用于探测并收集环境的三维信息；
建模模块，包括场景模拟器、对象模拟器，分别生成虚拟场景模型、虚拟对象模型；所述场景模拟器基于环境信息收集器收集的三维信息生成所述虚拟场景模型；所述虚拟对象模型具有空间属性和形态属性，所述空间属性包括模拟对象的空间位置和/或角度、所述形态属性包括模拟对象的自身形态和\或与其他对象或场景的连接关系；
显示模块，基于虚拟对象模型在虚拟场景模型中的定位，在现实场景上叠加显示所述虚拟对象模型、或在现实场景上叠加显示虚拟场景模型及虚拟对象模型；
输入模块，供用户操控所述虚拟对象模型或虚拟工具模型，以改变所述模拟对象的空间属性和\或形态属性。
进一步的，所述对象模拟器基于环境信息收集的三维信息，生成所述虚拟对象模型。
进一步的，所述家居设计系统存储有、或者连接到数据库，所述对象模拟 器从所述数据库中获取信息并生成虚拟对象模型。
进一步的，所述形态属性包括形状属性、大小属性、连接属性、装配属性中的至少一种。
进一步的，所述家居设计系统还包括工具模型器，所述工具模型器生成虚拟工具模型，所述虚拟工具模型能够模拟现实工具的加工效果，相应的改变模拟对象的形态属性。
进一步的，所述工具模型器从内部工具库、或者外部数据源获取虚拟工具模型的属性数据，所述属性数据包含工具的类型数据及工作参数数据，所述工具模拟模块根据所述属性数据模拟现实工具的加工效果。
进一步的，所述外部数据源为工具本身或云端工具数据库。
进一步的，所述环境信息收集器为深度摄像仪、结构光探测仪、激光雷达中的一种。
进一步的，还包括购买模块，所述购买模块生成和模拟对象有关的服务和或商品购买建议。
进一步的，所述购买模块基于用户所选择操控的模拟对象的形态属性，生成加工所述现实对象所需的工具或服务的购买建议。
本发明还提供了一种基于增强现实的家居设计系统和工具的组合，其包括前述任一条所述的家居设计系统，还包括工具，所述家居设计系统录入所述工具的属性数据，并基于工具的属性数据，改变虚拟对象模型的形态属性。
进一步的，所述家居设计系统包括工具信息读取器，以识别所述工具并录入所述工具的属性数据。
进一步的，所述家居设计系统根据用户对模拟对象的操控，生成工具使用指导步骤，并向用户显示。
进一步的，所述工具为电动工具。
本发明还提供了一种工具，其上设有工具属性标记，供前述的家居设计系统读取，以获得所述工具的属性数据。
进一步的，所述工具属性标记为标记图案，RFID标签或内置存储器。
进一步的，所述工具为电动工具。
本发明还提供了一种基于增强现实的家居设计方法，包括以下步骤：
探测并收集现实场景的场景数据，所述场景数据包括图像信息和深度信息；
根据场景数据，生成现实场景的虚拟场景模型；
生成用户可操控的虚拟对象模型，所述虚拟对象模型具有空间属性和形态属性，所述空间属性包括模拟对象的空间位置和/或角度、所述形态属性包括虚拟对象模型的自身形态和\或与其他对象或场景的连接关系；
基于虚拟对象模型在虚拟场景模型中的定位，在现实场景上叠加显示所述虚拟对象模型、或在现实场景上叠加显示虚拟场景模型及虚拟对象模型；
接收用户对所述虚拟对象模型的操控，改变所述虚拟对象模型的空间属性和或形态属性。
本发明还提供了一种基于增强现实的家居设计方法，包括以下步骤：
生成现实场景的虚拟场景模型；
生成用户可操控的虚拟对象模型，所述虚拟对象模型具有空间属性和形态属性，所述空间属性包括模拟对象的空间位置和/或角度、所述形态属性包括虚拟对象模型的自身形态和\或与其他对象或场景的连接关系；
基于虚拟对象模型在虚拟场景模型中的定位，在现实场景上叠加显示所述虚拟对象模型、或在现实场景上叠加显示虚拟场景模型及虚拟对象模型；
接收用户对所述虚拟对象模型的操控，改变所述虚拟对象模型的空间属性和或形态属性；
基于用户对所述虚拟对象模型的操控，提供服务/商品购买建议。
上述基于增强现实的家居设计系统，通过使用环境信息收集器收集用户家中的三维信息，并基于该三维信息建模，提高了增强现实家居设计系统对用户具体环境和需求的针对性。通过提供工具模拟器，实时模拟真实工具对家居设计改装工作的加工效果，增加了家居DIY工作的乐趣。
附图说明
图1为一实施例的家居设计系统的模块图；
1、控制器；3、场景模拟器；4、对象模拟器；5、工具模拟器；6、服务器；7、环境信息收集器；8、显示模块；9、输入模块
具体实施方式
如图1所示，一实施例的基于增强现实的家居设计系统包括控制器1，控制器1包括场景模拟器3、对象模拟器4、工具模拟器5；环境信息收集器7、显示模块8以及输入模块9。家居设计系统和云端的服务器6之间通信。
环境信息收集器5能够扫描环境的三维信息，并发送给控制器。例如，环境信息收集器扫描环境中的现实场景，将现实场景的三维信息发送给场景模拟器3，场景模拟器3基于接收到的信息生成虚拟场景模型。环境信息收集器也可扫描环境中的现实对象，将现实对象的三维信息发送给对象模拟器，对象模拟器4基于接收到的信息生成虚拟对象模型。环境信息收集器也可扫描环境中的现实工具，将现实工具的信息发送给对象模拟器，对象模拟器4基于接收到的信息生成虚拟对象模型。在此，场景构成对象所处的背景环境，对象为场景中的待加工、改变的标的物体，工具为加工、改变标的物体的器具。在一个例子中，场景为用户家中的客厅，对象为需要挂到墙壁上的相框，而工具为电钻。然而，对象模拟器和工具模拟器也可以基于其他信源，如服务器的发送的信息生成虚拟对象模型和虚拟工具模型。具体在后详述。
除了三维信息，环境信息收集器还可以收集环境中的图像信息，在该种情况下，基于前述的信息所生成的虚拟场景模型、虚拟对象模型为三维信息和图像信息的融合模型。
在一种实施例中，环境信息收集器为多目深度相机，多目深度相机至少包括两个摄像头，其利用了目标点在两个摄像头所拍摄的两幅视图上成像的坐标差异，和目标点到成像平面的距离之间的反比关系，来获取目标点的距离信息。在收集环境三维信息和建立模型时，首先通过多目深度相机扫描环境，然后，控制器中的相应建模模块(如场景建模器、对象建模器等)将照片分解成图片、以及对应的深度信息并记录下来。再后，通过内置的三维重建算法，建模模块将图片拼接为全景图，再将记录的深度信息还原到图片上，最终就得到了所扫描的物体的三维模型。
在另一种实施例中，环境信息收集器为结构光扫描仪。结构光是已知空间方向的投射光线，可以是各种激光、投影仪或者其他光源投射的点、直线、栅 格线。结构光扫描仪可以包括激光发射器和摄像头，例如，激光发射器以一定角度投射点状激光，反射点在摄像头的X轴向上的位置及对应反射点与测量装置的直线距离。如果所投射的为线状激光，经过结构光扫描仪360°扫描后，可以得到3D点云。在一种实施方案中，摄像头可以有多个。在一种实施方案中，结构光扫描仪包括相机以获取被扫描物体的明暗、色彩等信息。与多目深度相机的方案类似，结构光扫描仪获得了环境三维信息之后，控制器中的相应建模模块将图片、以及对应的深度信息并记录下来。再后，通过内置的三维重建算法，建模模块将图片拼接为全景图，再将记录的深度信息还原到图片上，最终就得到了所扫描的物体的三维模型。
在另一种可选的实施例中，环境信息收集器为测距仪，例如激光测距仪。测距仪360°旋转并扫描周围的物体，获得环境的三维信息并发送到控制器1，控制器1中的建模模块根据接收的信息，建立虚拟三维模型，例如虚拟场景模型、虚拟对象模型、虚拟工具模型等。
如前所述，控制器1包括场景模拟器，对象模拟器、工具模拟器。需要指出，上述各模拟器为非结构的功能模块。控制器1的实体包括微处理器、易失性存储器、非易失性存储器等习知的部件。非易失性存储器内存储有建模相关的算法。
场景模拟器用于接收场景信息以建立或更新虚拟场景模型。如前所述，在一种实施例中，场景信息来自于环境信息收集器所收集的场景三维信息和图像信息，例如客厅、书房等房间的信息。在其他实施例中，场景信息也可来自于云端的服务器，或者用户的输入等。虚拟场景模型包括场景的三维空间信息和表面图像信息；在一些实施例中，还包括场景中的物体的材质信息等。
对象模拟器用于接收对象信息以建立虚拟对象模型。在一种实施例中，对象信息来自于环境信息收集器所收集的对象的三维信息和图像信息。但在其他的实施例中，对象信息也可以来自于云端的服务器或者用户的输入等。例如，本系统中，待加工或改装的对象可以并不真实存在于用户家中，而是由用户在一个对象库中选取后，由对象模拟器建模并投射在虚拟场景模型中，或者现实场景中。例如，用户想在自己的客厅中试验增添一个茶几的效果，则可以先在 本地或者云端的对象库中选择一个茶几，通过对象模拟器建模之后，显示于虚拟场景模型或者现实场景中，以供用户观察茶几的尺寸是否合适，风格是否和现有的布置搭配等。用户再通过输入模块调整虚拟茶几模型的参数，如空间位置、颜色等，显示模块根据用户的输入调整显示内容，反映用户的操作。优选的，显示模块会根据用户的输入，实时调整显示内容。
在更复杂的实施例中，对象模拟器可以即根据环境信息收集器的信息建立一些虚拟对象模型，也从对象库中选取模型。仍以客厅为例，本系统由环境信息收集器扫描场景信息，供场景模拟器建立客厅的虚拟场景模型；扫描沙发、落地灯等对象信息，供对象模拟器建立虚拟沙发模型、虚拟落地灯模型，然后用户再从对象库中选择一个茶几，供对象模拟器建立虚拟茶几模型。随后，显示模块将虚拟场景模型和各个虚拟对象模型融合在一起，并通过显示模块向用户显示茶几的摆放效果，并且通过输入模块，供用户调整各个虚拟对象模型的参数并将调整后的效果再次通过显示模块显示。在本实施例中，用户既能观察现实场景中并不存在的对象、如茶几在场景中的摆放效果，还能调整现实场景中存在的对象、如沙发、落地灯等的摆放效果，从而可以观察和试验添置了家居用品之后，现有的家居用品需要如何挪动，挪动后空间是否足够，效果是否满意等。
对象模拟器建立的对象模型除了包括空间属性、颜色属性等外形相关的属性参数，还包括形态属性。形态属性和对象的可改装性质相关。例如，形态属性包括对象的可分解属性，例如是否可以拆成不同的块；或对象的可加工属性，任意位置或者特定位置是否可以切割或者打磨；或对象的安装属性，如是否可以和其他对象或者场景以某种方式安装或拼接，如粘合、焊接、镶嵌、螺纹连接等。
工具模拟器用于建立虚拟工具模型。前述的虚拟场景模型和虚拟对象模型均具有和现实相当物对应的三维尺寸信息。在一个实施例中，虚拟工具模型与之不同，可以不具有三维尺寸信息，但具有类型信息和加工/改装能力信息，例如。类型信息为该虚拟工具模型为电钻，或往复锯，砂光机等；加工/改装能力信息为电钻的功率、工作时间、加工孔径等；或往复锯的功率、可切割工件类 型等。当然，在其他实施例中，虚拟工具模型也可具有三维尺寸信息。
在一种实施例中，工具模拟器根据从服务器或本地存储器的数据库中获得的工具数据建立虚拟工具模型；在另一种实施例中，工具模拟器根据现实工具建立虚拟工具模型，例如，现实工具上具有可读取标志物如二维码、磁标或者RFID标签等，其中记录了工具信息，家居设计系统相应的具有扫码器，磁感应器或者RFID阅读器等来读取工具信息。工具模拟器根据读取的工具信息建立虚拟工具模型。进一步可选的，前述可读取的工具信息仅包括简单的工具类型信息，工具模拟器根据工具类型信息，从数据库中读取详细工具信息，如前述的加工/改装能力信息等。
在一种实施例中，显示模块为透明或者半透明的显示屏。该种显示屏的优势在于用户可以透过显示屏看到现实环境，从而将虚拟模型，以及其他需要的信息叠加显示在现实环境中。例如，显示模块可以为带有透明显示镜片的头盔或者眼睛。显示模块中内置有位置和姿态传感器，能够感应到自身位置姿态的变化以及和使用者的相对位置关系，从而保证显示的内容随用户视角而变化，始终处于正确位置。例如，在用户位置和姿态发生变化时，始终保证虚拟场景模型和现实场景重合，虚拟对象模型处于虚拟场景模型中、也即是现实场景中的设定位置。
在可选的实施例中，显示模块可以为手持式显示器，投影仪等。
在一种实施例中，输入模块可以为触摸板，用户通过触控操作进行输入。在其他实施例中，输入模块可以为自然交互界面设备，例如手势感应设备，感应用户的自然手势进行相应的控制，例如用户的拖拽、推移等动作。在更进一步的实施例中，输入设备和显示模块相互融合，例如，用户通过自然手势控制显示设备所显示在虚拟空间中的虚拟对象和虚拟工具，执行用户的各种指令。例如，用户可以直接拖移虚拟空间中的虚拟茶几，其就会相应移动到所拖动的位置，用户也可以用虚拟的电钻将虚拟相框挂在某个墙面。
在一种实施例中，家居设计系统模拟现实工具对现实对象的加工效果。在模拟之前，家居设计系统的各个模拟器首先完成建模工作，例如，建立虚拟场景模型、虚拟工具模型和虚拟对象模型。建模完成后，显示模块在用户视野中， 叠加于真实场景显示虚拟工具模型和虚拟对象模型，用户通过前述的操作模块操作虚拟工具模型，随着用户的操作，显示模块实时显示加工过程，并在加工完成后显示加工结果。在一种具体场景中，用户可以操作一个虚拟曲线锯，在虚拟木板上切割出一个圆形板。在一种具体场景中，虚拟对象模型叠加显示于真实对象上，在用户操作虚拟工具模型时，加工过程将实时叠加显示于真实对象之上。在上述的操作过程中，虚拟对象模型的属性被改变，例如，虚拟木板的形状属性由方形变为圆形。
在一种实施例中，家居设计系统为用户提供商品/服务供应建议。如前所述的，家居设计系统内置本地数据库或与云端数据库通信，获取商品/服务供应信息。商品/服务供应信息包括但不限于，虚拟对象模型所对应的现实物品的购买或出租信息，如沙发、相框等的购买链接或购买地址、售价或租金等；虚拟工具模型所对应的现实工具的购买或出租信息、如电钻、往复锯，螺丝批等工具的购买链接或购买地址、售价或租金等。例如，在用户进入家居设计系统，进行家居设计时，若用户选择在家中添置摆设物品，并从数据库中选取了虚拟对象模型时，家居设计系统同时从数据库中获取虚拟对象模型对应的物品的购买链接，并通过显示模型显示给用户，用户可以通过输入模块改变虚拟对象模型的位置、切换其他款型的虚拟对象模型等，如果效果满意，则可以通过商品/服务供应信息来进行购买，例如在家居设计系统中直接下单和付款，由商家送货上门。用户也可以在通过家居设计系统进行家居设计时，购买或者租借所选中的虚拟工具模型对应的现实工具，具体过程类似上面的物品购买过程，不再赘述。
用户也可以在通过家居设计系统进行家居设计时，购买或获得家居设计/改装服务。例如，用户在通过家居设计系统，选择了需要购买的商品，并确定了摆设、改装方案之后，家居设计系统直接将整体设计方案打包生成一个购买链接，用户确认购买后，商家提供商品送货上门和摆设、改装服务。具体的，比如用户在家居设计系统中，通过增强现实技术确认了在客厅中安放一套特定款型的沙发、在墙面上设定的位置悬挂若干幅画，并铺设地毯。那么，该整体设计方案将被上传到商家，商家按照整体设计方案将沙发、画、地毯送到客户家 中，并安装设计方案，将地毯铺好、沙发摆放到位、画安装到墙上。
在一种实施例中，家居设计系统为用户提供设计/改装操作指导建议。某些改装工作涉及到的工序比较多，并且需要用到多种工具，那么在这种典型情形下，家居设计系统能够为用户一步一步的分解操作指导。例如，用户需要自己动手制作一个书柜，在设计方案确定、原材料准备完成之后，家居设计系统通过显示模块指导用户进行工作，显示模块如前所述，可以为投影仪或带有透明或半透明显示仪的眼镜或头盔，显示模块将操作指导直接叠加显示在现实场景、对象和工具上。具体的，书柜制作需要板材切割、板材抛光、表面喷漆、安装孔制作、拼接固定等步骤；需要使用到夹具、切割锯、砂光机、刷子或喷漆设备、电钻等工具。相应的，家居设计系统分解为多个操作步骤，并在显示模块上显示目前正在进行的步骤，需要使用的材料、工具，以及需要用户如何操作。例如，第一步为，将板材夹持在夹具上，那么显示模块则在板材和夹具的位置上做出标记，并通过动画或者文字告诉用户将板材夹在夹具上；第二步为，使用切割锯切割出定长的板材，那么显示模块在用户完成第一步后，取消第一步中的提示，改为在切割锯上做出标记，并在夹具上的板材的待切割位置显示一条虚拟切割写，并通过动画或者问题告诉用户使用切割锯，沿切割线切割板材。后续的分解步骤，家居设计系统的指导方式类似，不再赘述。

Claims (19)

  1. 一种基于增强现实的家居设计系统，其特征在于，包括：
    环境信息收集器，用于探测并收集环境的三维信息；
    建模模块，包括场景模拟器、对象模拟器，分别生成虚拟场景模型、虚拟对象模型；所述场景模拟器基于环境信息收集器收集的三维信息生成所述虚拟场景模型；所述虚拟对象模型具有空间属性和形态属性，所述空间属性包括模拟对象的空间位置和/或角度、所述形态属性包括模拟对象的自身形态和\或与其他对象或场景的连接关系；
    显示模块，基于虚拟对象模型在虚拟场景模型中的定位，在现实场景上叠加显示所述虚拟对象模型、或在现实场景上叠加显示虚拟场景模型及虚拟对象模型；
    输入模块，供用户操控所述虚拟对象模型，以改变所述模拟对象的空间属性和\或形态属性。
  2. 根据权利要求1所述的家居设计系统，其特征在于，所述对象模拟器基于环境信息收集的三维信息，生成所述虚拟对象模型。
  3. 根据权利要求1所述的家居设计系统，其特征在于：所述家居设计系统存储有、或者连接到数据库，所述对象模拟器从所述数据库中获取信息并生成虚拟对象模型。
  4. 根据权利要求1所述的家居设计系统，其特征在于，所述形态属性包括形状属性、大小属性、连接属性、装配属性中的至少一种。
  5. 根据权利要求1所述的家居设计系统，其特征在于：所述家居设计系统还包括工具模型器，所述工具模型器生成虚拟工具模型，所述虚拟工具模型能够模拟现实工具的加工效果，相应的改变模拟对象的形态属性。
  6. 根据权利要求5所述的家居设计系统，其特征在于：所述工具模型器从内部工具库、或者外部数据源获取虚拟工具模型的属性数据，所述属性数据包含工具的类型数据及工作参数数据，所述工具模拟模块根据所述属性数据模拟现实工具的加工效果。
  7. 根据权利要求6所述的家居设计系统，其特征在于，所述外部数据源为工具本身或云端工具数据库。
  8. 根据权利要求1所述的家居设计系统，其特征在于：所述环境信息收集器为深度摄像仪、结构光探测仪、激光雷达中的一种。
  9. 根据权利要求1所述的家居设计系统，其特征在于：还包括购买模块，所述购买模块生成和模拟对象有关的服务和或商品购买建议。
  10. 根据权利要求9所述的家居设计系统，其特征在于：所述购买模块基于用户所选择操控的模拟对象的形态属性，生成加工所述现实对象所需的工具或服务的购买建议。
  11. 一种家居设计系统和工具的组合，包括前述任一权利要求所述的家居设计系统，还包括工具，其特征在于，所述家居设计系统录入所述工具的属性数据，并基于工具的属性数据，改变虚拟对象模型的形态属性。
  12. 根据权利要求11所述的组合，其特征在于：所述家居设计系统包括工具信息读取器，以识别所述工具并录入所述工具的属性数据。
  13. 根据权利要求11所述的组合，其特征在于，所述家居设计系统根据用户对模拟对象的操控，生成工具使用指导步骤，并向用户显示。
  14. 根据权利要求11所述的组合，其特征在于，所述工具为电动工具。
  15. 一种工具，其特征在于，其上设有工具属性标记，供权利要求11所述的家居设计系统读取，以获得所述工具的属性数据。
  16. 根据权利要求15所述的工具，其特征在于，所述工具属性标记为标记图案，RFID标签或内置存储器。
  17. 根据权利要求15所述的工具，其特征在于，所述工具为电动工具。
  18. 一种基于增强显示的家居设计方法，其特征在于，包括以下步骤：
    探测并收集现实场景的场景数据，所述场景数据包括图像信息和深度信息；
    根据场景数据，生成现实场景的虚拟场景模型；
    生成用户可操控的虚拟对象模型，所述虚拟对象模型具有空间属性和形态属性，所述空间属性包括模拟对象的空间位置和/或角度、所述形态属性包括虚拟对象模型的自身形态和\或与其他对象或场景的连接关系；
    基于虚拟对象模型在虚拟场景模型中的定位，在现实场景上叠加显示所述虚拟对象模型、或在现实场景上叠加显示虚拟场景模型及虚拟对象模型；
    接收用户对所述虚拟对象模型的操控，改变所述虚拟对象模型的空间属性和或形态属性。
  19. 一种基于增强显示的家居设计方法，其特征在于，包括以下步骤：
    生成现实场景的虚拟场景模型；
    生成用户可操控的虚拟对象模型，所述虚拟对象模型具有空间属性和形态属性，所述空间属性包括模拟对象的空间位置和/或角度、所述形态属性包括虚拟对象模型的自身形态和\或与其他对象或场景的连接关系；
    基于虚拟对象模型在虚拟场景模型中的定位，在现实场景上叠加显示所述虚拟对象模型、或在现实场景上叠加显示虚拟场景模型及虚拟对象模型；
    接收用户对所述虚拟对象模型的操控，改变所述虚拟对象模型的空间属性和或形态属性；
    基于用户对所述虚拟对象模型的操控，提供服务/商品购买建议。
PCT/CN2017/113605 2016-11-29 2017-11-29 基于增强现实的家居设计系统和方法 Ceased WO2018099400A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201611075207 2016-11-29
CN201611075207.0 2016-11-29

Publications (1)

Publication Number Publication Date
WO2018099400A1 true WO2018099400A1 (zh) 2018-06-07

Family

ID=62241235

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/113605 Ceased WO2018099400A1 (zh) 2016-11-29 2017-11-29 基于增强现实的家居设计系统和方法

Country Status (1)

Country Link
WO (1) WO2018099400A1 (zh)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109656362A (zh) * 2018-12-14 2019-04-19 歌尔科技有限公司 虚拟增强现实设备及其控制方法、系统、计算机存储介质
CN110443897A (zh) * 2019-08-02 2019-11-12 广州彩构网络有限公司 一种基于vr虚拟技术的室内家装系统
CN110728756A (zh) * 2019-09-30 2020-01-24 亮风台（上海）信息科技有限公司 一种基于增强现实的远程指导的方法与设备
CN111046454A (zh) * 2019-12-20 2020-04-21 武昌首义学院 一种三维虚拟室内设计仿真系统
CN111182278A (zh) * 2018-11-09 2020-05-19 上海云绅智能科技有限公司 一种投影展示管理方法及系统
CN113138660A (zh) * 2020-01-17 2021-07-20 北京小米移动软件有限公司 信息的获取方法、装置、移动终端及存储介质
CN114596140A (zh) * 2022-03-11 2022-06-07 合肥工业大学 一种面向开发人员的用户体验ar商品场景的重建系统
CN115296947A (zh) * 2022-06-28 2022-11-04 青岛海尔科技有限公司 控制命令的响应方法和装置、存储介质及电子装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104318042A (zh) * 2014-11-20 2015-01-28 江苏中佑石油机械科技有限责任公司 家居设计系统
CN104866691A (zh) * 2015-06-18 2015-08-26 上海斐讯数据通信技术有限公司 家居设计系统及家居设计方法
CN105069219A (zh) * 2015-07-30 2015-11-18 渤海大学 一种基于云设计的家居设计系统
CN105787230A (zh) * 2016-05-26 2016-07-20 河南联加网络科技有限公司 一种家居仿真设计系统及方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104318042A (zh) * 2014-11-20 2015-01-28 江苏中佑石油机械科技有限责任公司 家居设计系统
CN104866691A (zh) * 2015-06-18 2015-08-26 上海斐讯数据通信技术有限公司 家居设计系统及家居设计方法
CN105069219A (zh) * 2015-07-30 2015-11-18 渤海大学 一种基于云设计的家居设计系统
CN105787230A (zh) * 2016-05-26 2016-07-20 河南联加网络科技有限公司 一种家居仿真设计系统及方法

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111182278A (zh) * 2018-11-09 2020-05-19 上海云绅智能科技有限公司 一种投影展示管理方法及系统
CN109656362A (zh) * 2018-12-14 2019-04-19 歌尔科技有限公司 虚拟增强现实设备及其控制方法、系统、计算机存储介质
CN110443897A (zh) * 2019-08-02 2019-11-12 广州彩构网络有限公司 一种基于vr虚拟技术的室内家装系统
CN110728756A (zh) * 2019-09-30 2020-01-24 亮风台（上海）信息科技有限公司 一种基于增强现实的远程指导的方法与设备
CN110728756B (zh) * 2019-09-30 2024-02-09 亮风台（上海）信息科技有限公司 一种基于增强现实的远程指导的方法与设备
CN111046454A (zh) * 2019-12-20 2020-04-21 武昌首义学院 一种三维虚拟室内设计仿真系统
CN111046454B (zh) * 2019-12-20 2024-01-12 武昌首义学院 一种三维虚拟室内设计仿真系统
CN113138660A (zh) * 2020-01-17 2021-07-20 北京小米移动软件有限公司 信息的获取方法、装置、移动终端及存储介质
CN114596140A (zh) * 2022-03-11 2022-06-07 合肥工业大学 一种面向开发人员的用户体验ar商品场景的重建系统
CN115296947A (zh) * 2022-06-28 2022-11-04 青岛海尔科技有限公司 控制命令的响应方法和装置、存储介质及电子装置
CN115296947B (zh) * 2022-06-28 2024-01-26 青岛海尔科技有限公司 控制命令的响应方法和装置、存储介质及电子装置

Similar Documents

Publication Publication Date Title
CN109840953B (zh) 基于增强现实的家居设计系统和方法
US12271994B2 (en) Virtual interaction with three-dimensional indoor room imagery
US12597212B2 (en) Project management system with client interaction
US11244223B2 (en) Online garment design and collaboration system and method
CN108492356A (zh) 增强现实系统及其控制方法
US10049500B2 (en) Augmented reality e-commerce for home improvement
Hořejší Augmented reality system for virtual training of parts assembly
US7039486B2 (en) Method and device for viewing, archiving and transmitting a garment model over a computer network
US20150325038A1 (en) Presenting realistic designs of spaces and objects
US11948057B2 (en) Online garment design and collaboration system and method
TW201901366A (zh) 在構建、改造和製造中的增強現實任務辨識和協助
KR101813394B1 (ko) 하우스 인테리어 서비스 제공 시스템 및 방법
JP7208549B2 (ja) 仮想空間制御装置、その制御方法、及びプログラム
TW201714137A (zh) 立體虛擬實境的互動房屋瀏覽方法及其系統
US8854362B1 (en) Systems and methods for collecting data
CN106683177A (zh) 基于互动漫游式房屋装修数据交互方法及装置
Sharika et al. Interactive interior design using augmented reality and 3D modeling
JPH10188040A (ja) 不透明スクリーン式表示装置
Nandakumar et al. An in-depth evaluation of ar-based interior design and decoration applications
Sundaram et al. Plane detection and product trail using augmented reality
Straub et al. A very low-cost 3D scanning system for whole-body imaging
CN114445171A (zh) 产品展示方法、装置、介质及vr设备
Pentenrieder Augmented reality based factory planning
Manuri et al. A workflow analysis for implementing AR-based maintenance procedures
JP2002109307A (ja) シミュレーションシステムとシミュレーション方法及びコンピュータ読み取り可能な記録媒体

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17875409

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17875409

Country of ref document: EP

Kind code of ref document: A1