WO2020067204A1 - Dispositif de création de données d'apprentissage, procédé de génération de modèle d'apprentissage automatique, dispositif de création de données d'apprentissage et programme - Google Patents

Dispositif de création de données d'apprentissage, procédé de génération de modèle d'apprentissage automatique, dispositif de création de données d'apprentissage et programme Download PDF

Info

Publication number
WO2020067204A1
WO2020067204A1 PCT/JP2019/037684 JP2019037684W WO2020067204A1 WO 2020067204 A1 WO2020067204 A1 WO 2020067204A1 JP 2019037684 W JP2019037684 W JP 2019037684W WO 2020067204 A1 WO2020067204 A1 WO 2020067204A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
learning data
data
virtual
data creation
Prior art date
Application number
PCT/JP2019/037684
Other languages
English (en)
Japanese (ja)
Inventor
叡一 松元
颯介 小林
悠太 菊池
祐貴 五十嵐
統太郎 中島
Original Assignee
株式会社Ｐｒｅｆｅｒｒｅｄ Ｎｅｔｗｏｒｋｓ
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社Ｐｒｅｆｅｒｒｅｄ Ｎｅｔｗｏｒｋｓ filed Critical 株式会社Ｐｒｅｆｅｒｒｅｄ Ｎｅｔｗｏｒｋｓ
Publication of WO2020067204A1 publication Critical patent/WO2020067204A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present invention relates to a learning data creation method, a machine learning model generation method, a learning data creation device, and a program.
  • Semantic segmentation is a task of classifying each pixel (pixel) in an image captured by a camera device or the like into a class (for example, an object name of an object represented by the pixel) according to the meaning indicated by the pixel. .
  • a machine learning model is often learned by supervised learning.
  • learning data used for supervised learning is often created manually.
  • teacher information class classification of each pixel
  • Learning data is created.
  • learning data to which a plurality of pieces of teacher information are added. For example, in addition to the above-described class classification, learning data to which the posture of an object in an image (the direction and rotation of the object, etc.), the state of the object, and the like are added as teacher information may be necessary.
  • an embodiment of the present invention is directed to a first embodiment in which first data recording a first object in the real world and the first data generated by a simulator and aligned with the first object are arranged. And the second data that records the second object corresponding to the first object, at least the information based on the second data or the information generated by the simulator with respect to the first data.
  • One or a plurality of computers perform the assignment of one of them as teacher information.
  • FIG. 1 is a diagram illustrating an example of an overall configuration of a learning data creation system according to an embodiment of the present invention.
  • FIG. 9 is a diagram for schematically explaining an example of creation of learning data.
  • FIG. 6 is a diagram for explaining an example of a flow of a preparation procedure.
  • FIG. 9 is a diagram for describing an example of the flow of a learning data creation procedure. It is a figure showing an example of a teacher information list.
  • FIG. 2 is a diagram illustrating an example of a hardware configuration of a learning data creation device according to an embodiment of the present invention.
  • the predetermined task includes, for example, recognition and classification of an object in an image captured by a camera device or the like, grasping of the state of the object, and any action related to the object (for example, an object grasping action or an object avoiding action). ) And other tasks.
  • the object is an example of a target to which teacher information is added.
  • an image obtained by photographing a virtual space created by a three-dimensional simulator with a camera device that is, a virtual camera device installed in the virtual space or the like
  • a camera device that is, a virtual camera device installed in the virtual space or the like
  • an image hereinafter, also referred to as an “actual photographed image” of a real space corresponding to the virtual space (that is, a real space) photographed by an actual camera device.
  • learning data is created by adding teacher information obtained from the virtual photographed image.
  • the teacher information for example, contour information of an object in a virtual captured image, a class to which the object is classified, an object name of the object, state information of the object, a depth to the object, a depth to the object, a posture of the object, Information for performing a predetermined action on the object is included.
  • the virtual captured image is an example of data that records an object (a second target) captured in the virtual space, and is an example of second data.
  • the actual photographed image is an example of data recording an object (first object) photographed in the real space, and is an example of first data.
  • a three-dimensional simulator is an example of a simulator.
  • the real space corresponding to the virtual space is, for example, a real space in which the same object is arranged at the same position as the virtual space created by the three-dimensional simulator.
  • the position being the same in the virtual space and the real space means that, for example, when the same coordinate system is set in the virtual space and the real space, the position coordinates are the same.
  • coordinate systems that can be mutually converted may be set.
  • position and “posture” represent a position and a posture in the same coordinate system set in the virtual space and the real space.
  • the same or the same is not limited to exactly the same or the same.
  • the same or the same may be allowed according to a shift generated due to an error in an apparatus or a calculation, a purpose of using a data set, or the like. It is assumed that deviations can be included.
  • the object is the same in the virtual space and the real space means that the object represented by the three-dimensional model arranged in the virtual space is the same as the real object arranged in the real space. That is.
  • an object arranged in a virtual space is also referred to as an “object” in order to distinguish it from an actual object arranged in a real space.
  • the fact that the target or the object is the same means that the information necessary for providing the teacher information is the same. For example, in semantic segmentation, an object having the same contour is included, and in teaching a gripping position of the object, an object having the same tactile sensation and material information in addition to the contour is included.
  • FIG. 1 is a diagram illustrating an example of an overall configuration of a learning data creation system 1 according to an embodiment of the present invention.
  • the learning data creation system 1 includes a learning data creation device 10, one or more camera devices 20, and one or more tracking devices 30.
  • the learning data creation device 10, the camera device 20, and the tracking device 30 are communicably connected via a communication network such as a wireless LAN (Local Area Network).
  • This communication network may be, for example, a wire LAN or the like in whole or in part.
  • the learning data creation device 10 is a computer or a computer system that creates learning data.
  • the learning data creation device 10 includes a three-dimensional simulator 100, a learning data creation unit 200, and a storage unit 300.
  • the three-dimensional simulator 100 of the present embodiment is a simulator that can simulate a three-dimensional virtual space.
  • an object can be arranged in a virtual space, or a physical operation for simulating the physical laws of the object (for example, collision determination between objects) can be performed.
  • a virtual photographed image photographed in a virtual space with a virtual camera device can be drawn.
  • the outline information and the object name of the object (object) in the virtual captured image, the class to which the object is classified, the state information of the object, the depth to the object, the depth to the object, , The result of performing a predetermined physical operation on the object, and the like can be added to the virtual captured image.
  • These pieces of information are generated by calculation or the like in the three-dimensional simulator 100. Note that these pieces of information may be generated in advance before the virtual captured image is drawn, for example, and separately stored in the storage unit 300 or the like.
  • a data set can be easily created by adding the information generated by the simulator to the first data.
  • Such a three-dimensional simulator 100 is realized by a game engine such as, for example, Unity, Unreal @ Engine4 (UE4), Blender, and the like.
  • a game engine such as, for example, Unity, Unreal @ Engine4 (UE4), Blender, and the like.
  • UE4 Unreal @ Engine4
  • Blender Blender
  • the three-dimensional simulator 100 is not limited to these game engines, and may be realized by any three-dimensional simulation software.
  • the learning data creation unit 200 creates learning data by adding teacher information obtained from a virtual captured image to a real captured image.
  • the teacher information is information added to the virtual captured image when the three-dimensional simulator 100 draws the virtual captured image (for example, contour information and an object name of an object in the virtual captured image, a class into which the object is classified). , State information of the object, a depth to the object, a posture of the object, a result of performing a predetermined physical operation on the object, and the like).
  • the learning data creating unit 200 creates learning data by adding information obtained from the virtual captured image drawn by the three-dimensional simulator 100 to the real captured image as teacher information.
  • the storage unit 300 stores various information.
  • the information stored in the storage unit 300 includes, for example, an actual captured image captured by the camera device 20 in the real space, a virtual captured image drawn by the three-dimensional simulator 100, tracking information acquired from the tracking device 30, Examples include a three-dimensional model of an object to be arranged.
  • the tracking information is information obtained by tracking the position and orientation of the camera device 20 in the real space. That is, the tracking information is information indicating both the position and the posture of the camera device 20 at each time. However, the tracking information may be, for example, information indicating only the position of the camera device 20 at each time.
  • the tracking information obtained by the tracking device 30 is used for synchronizing information in the real space and information in the virtual space.
  • the camera device 20 is an imaging device that captures an image of a real space and creates an actually captured image.
  • the camera device 20 is fixed to, for example, a portable camera stand on which the tracking device 30 is mounted.
  • the actual captured image created by the camera device 20 is transmitted to the learning data creation device 10 and stored in the storage unit 300.
  • the camera device 20 may be, for example, a depth camera that can create an actual captured image to which depth information has been added.
  • the tracking device 30 is a device that tracks the position and orientation of the camera device 20 and creates tracking information (for example, a sensing device equipped with a position sensor and an orientation sensor).
  • the tracking device 30 is mounted on, for example, a portable camera stand or the like. As described above, one tracking device 30 is installed and associated with one camera device 20.
  • the tracking information created by the tracking device 30 is transmitted to the learning data creation device 10 and stored in the storage unit 300.
  • the tracking device 30 may be directly mounted on the camera device 20 or may be built in the camera device 20, for example.
  • the tracking device 30 of the present embodiment is a device that acquires information when the first data is acquired, and can easily create the second data based on the information.
  • the configuration of the learning data creation system 1 shown in FIG. 1 is an example, and another configuration may be used.
  • the learning data creation system 1 may include an arbitrary number of camera devices 20 and tracking devices 30 corresponding to the camera devices 20.
  • the camera device 20 may not have the tracking device 30 corresponding to the camera device 20 as long as the position and the posture in the real space are known. For example, when the camera device 20 is fixedly installed at a predetermined position in a predetermined posture, the tracking device 30 corresponding to the camera device 20 may not be provided.
  • FIG. 2 is a diagram for schematically explaining an example of creating learning data.
  • an actual photographed image photographed by the camera device 20 in a certain posture at a certain position in the real space is referred to as “real photographed image G110”.
  • a virtual captured image captured by the virtual camera device having the same posture at the same position in the virtual space corresponding to the real space is referred to as a “virtual captured image G210”.
  • information for example, “contour line” and “object name” in FIG. 2 generated by calculation of the three-dimensional simulator 100 is added to the virtual captured image G210. That is, in the example shown in FIG. 2, the outline of each object in the virtual captured image G210 and the object name of each object are given. Note that, among information that can be generated by calculation of the three-dimensional simulator, what information is given to the virtual captured image G210 depends on the task to be executed by the machine learning model (that is, what learning data set Depends on what you want to create.).
  • the learning data creating device 10 adds the information (that is, the “contour line” and the “object name”) assigned to the virtual captured image G210 to the real captured image G110 as teacher information.
  • the data G120 is created.
  • learning data G120 represented by a set of the real photographed image G110 and the teacher information that is, “contour line” and “object name”.
  • the learning data creating apparatus 10 includes the real photographed image G110 actually photographing a certain area in the real space and the virtual photographing image G110 virtually photographing the same area in the virtual space. Using the captured image G210 and the information obtained from the virtual captured image G210 (that is, information generated by the operation of the three-dimensional simulator or the like) is added to the real captured image G110 to create the learning data G120. . Therefore, the learning data creation device 10 according to the embodiment of the present invention can easily create the learning data G120.
  • the position and orientation of the camera device 20 are specified by the tracking information acquired from the tracking device 30, so that the position of the virtual camera device in the virtual space is determined. And the posture can be synchronized with the camera device 20. For this reason, the user can easily obtain the real photographed image and the virtual photographed image corresponding to the real photographed image simply by photographing the real space with the camera device 20, for example.
  • FIG. 3 is a diagram for explaining an example of the flow of the advance preparation procedure.
  • Step S101 The three-dimensional simulator 100 acquires a three-dimensional model of an object placed in the virtual space from the storage unit 300. This means, for example, that the data of the three-dimensional model stored in the storage unit 300 is imported into the three-dimensional simulator 100.
  • the three-dimensional model is provided with information such as an object ID, an object name, and a category to which the object belongs, in addition to the shape of the object.
  • the three-dimensional model may be created in advance by an arbitrary method and stored in the storage unit 300.
  • a method of creating a three-dimensional model for example, the three-dimensional shape of an actual object may be created by scanning with a three-dimensional scanner or the like, or may be created manually with three-dimensional model creation software or the like. good.
  • Step S102 The three-dimensional simulator 100 arranges the object represented by the three-dimensional model in the virtual space.
  • the user selects a desired three-dimensional model from the plurality of three-dimensional models imported in step S101, and drags and drops the selected three-dimensional model into the virtual space.
  • An object can be arranged in the virtual space.
  • the user may be able to arrange the object represented by the three-dimensional model in the virtual space by designating the position coordinates in the virtual space.
  • the user when arranging the object represented by the three-dimensional model in the virtual space, the user may arbitrarily tilt or rotate the object, and then arrange the object. In addition, the user may arrange the object after enlarging or reducing the object, for example.
  • step S102 may be repeatedly performed.
  • the three-dimensional simulator 100 creates a virtual space in which one or more objects (objects) are arranged at desired positions.
  • Step S103 The user places an actual object in the real space so as to correspond to the virtual space created in steps S101 to S102.
  • the user can superimpose and display the object arranged in the virtual space in the real space, and use the display device equipped with the position sensor and the posture sensor to display the object. What is necessary is just to arrange the actual object at the same position as the object displayed on the device.
  • Examples of such a display device include a head-mounted display on which a position sensor, an attitude sensor, and a camera are mounted, a head-mounted display on which a real space is transparently visible and on which a position sensor and an attitude sensor are mounted, and projection mapping.
  • Examples include a device, a tablet terminal equipped with a position sensor, a posture sensor, and a camera, and a smartphone equipped with a position sensor, a posture sensor, and a camera.
  • These display devices can display a video in which an object is superimposed in the real space after synchronizing the position in the virtual space with the position in the real space. Therefore, for example, the user moves the display device in the real space after carrying or wearing the display device, and places the same object in the same position, the same posture, and the same position as the object in the video in the real space. Can be.
  • the virtual space created in steps S101 to S102 can be associated with the real space.
  • the same position as the object arranged in the virtual space is obtained.
  • the same object may be arranged in the real space in the posture.
  • FIG. 4 is a diagram for explaining an example of the flow of the learning data creation procedure.
  • Step S201 The user uses the camera device 20 to photograph a desired range in the real space. As a result, an actual photographed image is created by the camera device 20 and transmitted to the learning data creating device 10. In the learning data creation device 10, the actual photographed image is stored in the storage unit 300.
  • the tracking device 30 corresponding to the camera device 20 transmits the tracking information to the learning data creation device 10.
  • the tracking information is stored in the storage unit 300.
  • the tracking information is information indicating the position and orientation of the camera device 20 as described above.
  • the tracking information created by the tracking device 30 tracking the position and orientation of the camera device 20 is stored in the storage unit 300, but the present invention is not limited to this.
  • tracking information indicating the tracking result may be stored in the storage unit 300.
  • a two-dimensional code such as a QR code (registered trademark) is pasted on the camera device 20 in advance and the position and orientation of the camera device 20 are tracked by reading the two-dimensional code with a camera or the like. good.
  • Step S202 The three-dimensional simulator 100 photographs the virtual space with the virtual camera device at the same position and orientation as the camera device 20 photographed in step S201. In other words, the three-dimensional simulator 100 draws (renders) in the virtual space an imaging range of the virtual camera device having the same position and posture as the camera device 20 imaged in step S201.
  • the three-dimensional simulator 100 can specify the position and orientation of the camera device 20 from the tracking information created in step S201. Therefore, the three-dimensional simulator 100 can install a virtual camera device in the virtual space at the same position and orientation as the camera device 20 in the real space. As a result, a virtual captured image corresponding to the real captured image created in step S201 is created.
  • the three-dimensional simulator 100 adds predetermined information acquired or calculated in the virtual space to the virtual captured image. Then, the three-dimensional simulator 100 stores the virtual captured image to which the predetermined information is added in the storage unit 300.
  • the predetermined information for example, the outline information and the object name of the object (object) in the virtual captured image, the class to which the object is classified, the state information of the object, the , The posture of the object, the result of performing a predetermined physical operation on the object, and the like.
  • a result of performing a predetermined physical operation on an object for example, information on an operation of a robot arm capable of performing a preset operation and holding the object at the position can be given.
  • Step S202 may be automatically performed after step S201, for example, or may be performed in response to a user operation (for example, a rendering start operation in a virtual space). .
  • Step S203 The learning data creation unit 200 assigns, as teacher information, predetermined information given to the virtual shot image created in step S202 to the real shot image created in step S201. I do. As a result, learning data represented by a set of the real photographed image and the teacher information is created.
  • the teacher information included in the learning data is represented, for example, in a list format.
  • FIG. 5 illustrates a plurality of pieces of teacher information represented in a list format (this is also referred to as a “teacher information list”).
  • FIG. 5 is an example of a teacher information list assigned to a certain actually shot image (image ID: image101).
  • Each teacher information included in the teacher information list shown in FIG. 5 is information in which an object ID, position information, contour line information, contact information, and gripping operation information are associated with each other.
  • the object ID is an ID for identifying an object.
  • the object ID is, for example, information given to a three-dimensional model of an object arranged in a virtual space.
  • the position information is the position coordinates where the object is located.
  • the position information is, for example, information given to the object when the object represented by the three-dimensional model is arranged in step S102 described above.
  • Contour line information is information indicating a contour line of an object.
  • the contour information can be acquired, for example, from the rendering result when the virtual captured image is drawn (rendered) in step S202.
  • the contact information is information indicating the object ID of the other object, the contact position with the other object, and the like when the object with the object ID is in contact with another object (object).
  • the contact information can be obtained, for example, from the calculation result of the physical calculation of the three-dimensional simulator 100.
  • the gripping operation information is, for example, information relating to an operation in which a robot arm capable of performing a preset operation can grip an object having the object ID at a shooting position of a virtual shot image.
  • the gripping motion information can be obtained, for example, from the calculation result of the physical calculation of the three-dimensional simulator 100.
  • the teacher information list illustrated in FIG. 5 is a list of the teacher information in which the position information, the object name, the outline information, the contact information, and the grip operation information are associated with each other for each object (object). It is.
  • any information that can be obtained or calculated by the three-dimensional simulator 100 may be associated with the teacher information.
  • the virtual captured image itself or a partial region of the virtual captured image may be associated with the teacher information.
  • an image area portion representing the object of the object ID in the image area of the virtual captured image may be associated with the object ID.
  • the gripping motion information may include information on a target part that is considered to be grippable or easy to grip when gripping the target.
  • Each piece of teacher information may be information in which only a part of the above information (position information, object name, contour line information, contact information, gripping motion information, etc.) is associated.
  • teacher information included in the learning data is represented in a list format
  • teacher information included in the learning data may be represented in any other format.
  • FIG. 6 is a diagram illustrating an example of a hardware configuration of the learning data creation device 10 according to the embodiment of the present invention.
  • the learning data creating device 10 includes an input device 401, a display device 402, an external I / F 403, a communication I / F 404, and a random access memory (RAM). 405, a ROM (Read Only Memory) 406, a processor 407, and an auxiliary storage device 408. These pieces of hardware are interconnected by a bus 409.
  • the input device 401 is, for example, a keyboard, a mouse, a touch panel, or the like, and is used by a user to input various operations.
  • the display device 402 is, for example, a display or the like, and displays various processing results of the learning data creation device 10. Note that the learning data creation device 10 may not have at least one of the input device 401 and the display device 402.
  • the external I / F 403 is an interface with an external device.
  • the external device includes a recording medium 403a and the like. Reading and writing of the recording medium 403a and the like can be performed via the learning data creation device 10 and the external I / F 403.
  • the storage medium 403a may store one or more programs for realizing the three-dimensional simulator 100 and the learning data creation unit 200.
  • Examples of the recording medium 403a include a flexible disk, a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital Memory card), and a USB (Universal Serial Bus) memory card.
  • a flexible disk a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital Memory card), and a USB (Universal Serial Bus) memory card.
  • the communication I / F 404 is an interface for connecting the learning data creation device 10 to a communication network.
  • One or more programs that implement the three-dimensional simulator 100 and the learning data creation unit 200 may be obtained (downloaded) from a predetermined server device or the like via the communication I / F 404.
  • the RAM 405 is a volatile semiconductor memory that temporarily stores programs and data.
  • the ROM 406 is a nonvolatile semiconductor memory that can retain programs and data even when the power is turned off.
  • the ROM 406 stores, for example, settings related to an OS (Operating System), settings related to a communication network, and the like.
  • the processor 407 is, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like, and is an arithmetic device that reads a program or data from the ROM 406 or the auxiliary storage device 408 onto the RAM 405 and executes processing.
  • the three-dimensional simulator 100 and the learning data creation unit 200 are realized by, for example, processing that causes the processor 407 to execute one or more programs stored in the auxiliary storage device 408.
  • the learning data creation device 10 may include both the CPU and the GPU as the processor 407, or may include only one of the CPU and the GPU.
  • the auxiliary storage device 408 is, for example, a hard disk drive (HDD) or a solid state drive (SSD), and is a nonvolatile storage device that stores programs and data.
  • the auxiliary storage device 408 stores, for example, an OS, various application software, one or more programs for implementing the three-dimensional simulator 100 and the learning data creation unit 200, and the like.
  • the storage unit 300 is realized using, for example, the auxiliary storage device 408.
  • the storage unit 300 may be implemented using, for example, a storage device that is communicably connected to the learning data creation device 10 via a communication network, instead of the auxiliary storage device 408.
  • the learning data creation device 10 can implement the above-described various processes by having the hardware configuration shown in FIG. In the example illustrated in FIG. 6, a case has been described where the learning data creation device 10 according to the embodiment of the present invention is implemented by one device (computer), but the invention is not limited to this.
  • the learning data creation device 10 according to the embodiment of the present invention may be realized by a plurality of devices (computers).
  • the learning data creation system 1 converts information obtained from a virtual captured image (that is, information that can be obtained or calculated by the three-dimensional simulator 100) into a real captured image as teacher information. By giving, the learning data is created. For this reason, in the learning data creation system 1 according to the embodiment of the present invention, the learning data is easily created without, for example, manually adding teacher information to an actual captured image. Will be able to do it. In particular, for example, even when the number of teacher information is large (for example, the number of types of objects is large or the number of categories is large), the learning data can be easily created.
  • the object segmentation can be performed with high accuracy. Will be able to do it.
  • the teacher information is included. Learning data can be easily created.
  • the user simply captures an arbitrary range with the camera device 20 while moving in the real space, and obtains the real captured image and the real captured image. Is generated, so that a large amount of learning data can be easily created. For this reason, for example, a large amount of learning data can be obtained at low cost as compared with a case where teacher information is added to an actual captured image using crowd sourcing or the like.
  • the learning data creation system 1 for example, recognition of a robot that cleans a certain room that is a real space or clears an object in the room is performed. It is possible to easily obtain a large amount of learning data used for learning an engine (that is, a machine learning model that performs a task of cleaning and clearing a room).
  • a virtual space is created, and objects are arranged in the real space so as to correspond to the virtual space.
  • a virtual space may be created so as to correspond to a real space in which an object is actually arranged.
  • the object in the virtual space may be moved so as to correspond to the object in the real space.
  • a function for facilitating the alignment between the first target and the second target it is preferable to provide a function for facilitating the alignment between the first target and the second target.
  • a function may be provided that facilitates movement when moving an object in the virtual space, or facilitates alignment with an object in the real space.
  • a function of moving an object in the virtual space with reference to the position of the user for example, a function of drawing the object to the vicinity of the user
  • a function of aligning the normal with the real space for example, a predetermined function of the object in the virtual space
  • any of information such as the direction, position, and angle of the object in the virtual space
  • a function that aligns objects in the real space by manipulating other information after fixing the object a function that aligns objects in the real space with the corresponding objects in the virtual space using a machine learning model, etc.
  • the second object may be subjected to a process such that the second object satisfactorily corresponds to the first object or facilitates alignment. For example, when the object in the real space does not correspond to the object in the virtual space, or when it is difficult to match the position, an object in the virtual space is newly added, the information is corrected, or the information is generated again. You may do it.
  • data other than main data used as learning data may be recorded.
  • data is, specifically, the parameters of the device that acquires or creates the data (for example, in the case of a camera, the focal length, the position of the camera, the shutter speed and the aperture value, the acceleration sensor provided in the camera, Information on the camera at the time when the image was acquired, such as information on the movement of the camera, information on the environment in which data such as lighting conditions were acquired, category or tag information on the target or main data recorded in the data ( For example, in the case of an image, category information related to a photographed object) is included.
  • the main data is a moving image
  • time information and the like information used for management of recorded data and the like, for example, an ID of a worker or an imaging device may be included.
  • data other than the main data may be used in the process of creating the learning data, or may be used together with the main data as a part of the learning data.
  • the real photographed image and the virtual photographed image are still images, but the present invention is not limited to this.
  • the real photographed image and the virtual photographed image may be moving images.
  • a set of the real photographed image and the teacher information acquired from the three-dimensional simulator 100 is used as the learning data, but is not limited thereto.
  • a combination of a virtual captured image and teacher information may be used as learning data, using the real captured image as teacher information. In this case, it is possible to create learning data for a task of predicting an actual captured image from the virtual captured image created by the three-dimensional simulator 100.
  • Reference Signs List 1 learning data creation system 10 learning data creation device 20 camera device 30 tracking device 100 three-dimensional simulator 200 learning data creation unit 300 storage unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Processing Or Creating Images (AREA)

Abstract

Selon l'invention, en utilisant des premières données enregistrant un premier sujet réel et des secondes données enregistrant un second sujet qui est généré par un simulateur, est positionné en alignement avec le premier sujet et correspond au premier sujet, un ordinateur ou une pluralité d'ordinateurs ajoutent des informations basées sur les secondes données et/ou informations générées par le simulateur aux premières données en tant qu'informations d'apprentissage.
PCT/JP2019/037684 2018-09-27 2019-09-25 Dispositif de création de données d'apprentissage, procédé de génération de modèle d'apprentissage automatique, dispositif de création de données d'apprentissage et programme WO2020067204A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2018-182538 2018-09-27
JP2018182538A JP2022024189A (ja) 2018-09-27 2018-09-27 学習用データ作成方法、学習用データ作成装置及びプログラム

Publications (1)

Publication Number Publication Date
WO2020067204A1 true WO2020067204A1 (fr) 2020-04-02

Family

ID=69953491

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/037684 WO2020067204A1 (fr) 2018-09-27 2019-09-25 Dispositif de création de données d'apprentissage, procédé de génération de modèle d'apprentissage automatique, dispositif de création de données d'apprentissage et programme

Country Status (2)

Country Link
JP (1) JP2022024189A (fr)
WO (1) WO2020067204A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7281576B1 (ja) 2022-03-31 2023-05-25 Kddi株式会社 映像投影システム及び映像投影方法

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004362440A (ja) * 2003-06-06 2004-12-24 National Printing Bureau 印刷物の文字列抽出処理装置
WO2018020954A1 (fr) * 2016-07-29 2018-02-01 株式会社日立製作所 Système de construction de bases de données pour apprentissage automatique

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004362440A (ja) * 2003-06-06 2004-12-24 National Printing Bureau 印刷物の文字列抽出処理装置
WO2018020954A1 (fr) * 2016-07-29 2018-02-01 株式会社日立製作所 Système de construction de bases de données pour apprentissage automatique

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
FUJIHASHI, KAZUKI ET AL.: "Estimation of Number and Locations of Products in Pictures by Semi-Supervised Learning", PROCEEDINGS DVD OF THE 32ND ANNUAL CONFERENCE OF THE JAPANESE SOCIETY FOR ARTIFICIAL INTELLIGENCE , 2018, 5 June 2018 (2018-06-05), pages 1 - 3 *

Also Published As

Publication number Publication date
JP2022024189A (ja) 2022-02-09

Similar Documents

Publication Publication Date Title
Nebeling et al. The trouble with augmented reality/virtual reality authoring tools
JP6105092B2 (ja) 光学式文字認識を用いて拡張現実を提供する方法と装置
US8243061B2 (en) Image processing apparatus and method of controlling operation of same
JP2020535509A (ja) イメージ内のターゲットオブジェクトに自動的にアノテーションするための方法、装置およびシステム
WO2019041900A1 (fr) Procédé et dispositif de reconnaissance d'opération d'assemblage/de simulation d'assemblage dans un environnement de réalité augmentée
CN112154486B (zh) 用于多用户增强现实购物的系统和方法
US11436755B2 (en) Real-time pose estimation for unseen objects
CN110573992B (zh) 使用增强现实和虚拟现实编辑增强现实体验
JP2023103265A (ja) 制御装置、制御方法、及びプログラム
Nishino et al. 3d object modeling using spatial and pictographic gestures
KR20200136723A (ko) 가상 도시 모델을 이용하여 객체 인식을 위한 학습 데이터 생성 방법 및 장치
WO2020067204A1 (fr) Dispositif de création de données d'apprentissage, procédé de génération de modèle d'apprentissage automatique, dispositif de création de données d'apprentissage et programme
CN113129362A (zh) 一种三维坐标数据的获取方法及装置
JP7267068B2 (ja) 学習済みモデル生成装置、プログラム及び学習済みモデル生成システム
US20200226833A1 (en) A method and system for providing a user interface for a 3d environment
GB2555521A (en) Improved object painting through use of perspectives or transfers in a digital medium environment
JP7401245B2 (ja) 画像合成装置、画像合成装置の制御方法およびプログラム
CN115082648A (zh) 一种基于标志物模型绑定的ar场景布置方法及系统
Okamoto et al. Assembly assisted by augmented reality (A 3 R)
BARON et al. APPLICATION OF AUGMENTED REALITY TOOLS TO THE DESIGN PREPARATION OF PRODUCTION.
JP7045863B2 (ja) 情報管理システム、情報管理方法、及びプログラム
JP6967150B2 (ja) 学習装置、画像生成装置、学習方法、画像生成方法及びプログラム
JP6859763B2 (ja) プログラム、情報処理装置
JP6204781B2 (ja) 情報処理方法、情報処理装置、およびコンピュータプログラム
JP2021192230A5 (fr)

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19865869

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19865869

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP