TWM528481U - Systems and applications for generating augmented reality images - Google Patents

Abstract

The systems and applications for generating augmented reality (AR) images are disclosed. The system includes a processing module and a digital microscope module having a plurality of camera unit, and the processing module tracks, detects and parses the user's motions, commands or event triggered to generate the related control signals, the virtual objects and then the composed AR images according to the received instant images of the observed samples captured by the digital microscope module. Moreover, the processing module generates and outputs the AR images composed of the instant images and the user interface(UI), icons, objects and information related to the interactive applications while the display mode switch or the real-time tutorial and sharing is triggered.

Description

Augmented Reality Image Generation System and Its Application

The present invention relates to a system for interacting with a user to be a microscopic object and an application thereof, and more particularly to an augmented reality image system for generating and manipulating a microscopic object and an application thereof.

With the increase of computer system computing and processing capabilities, as well as the increasing visual, intuitive operation, quality, response or transmission time and interactivity of input and output data or images, many image multimedia materials have been presented. Plane or static, which is achieved by stereoscopic, dynamic virtual or Augmented Reality (AR) images. Augmented reality is an image technology derived from Virtual Reality (VR). The biggest difference between the two is that virtual reality is a virtual environment to simulate the real world, while augmented reality is There will be virtually no objects or scenes on the scene, by presenting the virtual objects in real life and displaying them in the designated real space, in other words, based on the real world, to augment the virtual information, The combination of "realistic environmental imagery" and "computer virtual imagery" enables users to obtain relevant information by augmenting the reality and to see for themselves the operation of virtual three-dimensional objects in the actual environment. Therefore, if you can increase the interactivity and reduce the latency or alignment error in the application of augmented reality technology, it will be more helpful for the system users to learn and use motivation.

Currently, known methods for augmenting reality image interactive related applications include mark/recognition tags and no marks, as well as optical see-through and video see-through. Augmented reality technology or classification. The so-called augmented reality technology with mark/recognition is to provide a computer-readable identification tag, such as an interactive card that is usually used, to design the content of the interactive card according to the application and function of the augmented reality. The user can use the camera or the mobile phone as a medium to read the information contained in the interactive card, and then superimpose the corresponding augmented reality image into the real world in the display, that is, the display will augment the three-dimensional reality. The image is displayed on the interactive card. However, in order to correctly identify the content of the interactive card, the appearance and size of the card must meet certain conditions, such as a rectangular shape, a continuous border (usually using a black or white border), or a mark in the border. The image cannot be rotated or symmetrical, so it is limited by the interactive card; the augmented reality technology of the markless technology stores the pre-established specific still image in the feature database, when it is in the display or image When a specific still picture is detected, the corresponding augmented feature can be superimposed as a virtual object on the screen. However, both of the above prior art techniques have limitations in that they cannot provide augmented reality effects for those whose original materials belong to dynamic or multimedia, and lack visual stereoscopic feeling.

As for the so-called optical penetrating technology, a translucent mirror is used to present a real environment, an image or an object, and a virtual environment or object is presented by reflecting the translucent mirror, and the image penetrating technology is a virtual environment. An image or object that is superimposed on a sequence of images of the real environment captured by the camera or camera. The advantage of using the former is that there is no display delay when displaying the image of the real environment for the user, but the alignment error and display delay are caused by the fact that the real and the virtual are not synchronized, and the luminosity is reduced by the translucent mirror; There is no non-synchronization during display-timing, so there is no alignment error and display delay. Late, but when the augmented reality image is displayed to the user or viewer, a display delay occurs.

In addition, for users who need to observe, learn or operate the object to be treated by a microscope device such as a conventional electronic or optical device, if it is necessary to consult books or reference materials or record observations and operation results, it is often necessary to be frequently Forced to leave the microscope head or machine, and may not be able to quickly find the required reference materials, or you must open other computer devices or search the web page or database to obtain the required non-planar, non-static multimedia materials, and Traditional microscope devices lack mechanisms such as sharing, evaluation, warning, two-way real-time interactive guidance, remote control and video streaming. They are inconvenient, inefficient, do not meet user needs, and lack visual and visual effects such as graphics or images. Friendly, unintuitive, and lack of interaction and sharing mechanisms can also inhibit or make it difficult to create learning motivation and limit application areas. However, even though it is desirable to perform the above activities using non-traditional or conventional optical, stereoscopic, and surgical microscope devices, it has not been seen so far that the object to be observed, learned, or manipulated is required to be microscopically manipulated. Shaped objects, non-simulated or trained prostheses, models (such as fake eyes used to train cataracts or eyeball surgery, or animal eyeballs, etc.), and can utilize instruments previously used in the field (eg, capsuloendrons, scissors, or lasers) The operation of the scalpel, etc.), rather than the simulated operation of the object, does not cause the environment or the operating machine to be wetted and contaminated by the biological living tissue, and can reduce the cost of specifically searching or ordering the living tissue material, and can correctly identify the device. A technical solution that reduces the complexity of the computational end and produces a good quality binocular stereoscopic augmented reality image.

One of the main purposes of this work is to provide an augmented reality image generation system to effectively improve the lack of interaction and augmented reality image technology of traditional microscope devices, which is inconsistent with and cannot meet the observation, learning and bidirectional of the object to be observed. The issue of interactive needs.

Another main purpose of this creation is to provide a microsurgery teaching or training system using an image penetrating technology to effectively expand the application field and interactivity of augmented reality technology.

Another main purpose of this creation is to provide an electronic component assembly training and detection system using an image penetration technology to effectively expand the application field and interaction of the augmented reality technology.

Another main purpose of this creation is to provide an object microscopic observation and interaction system using an image penetrating technology to effectively expand the application field and interaction of augmented reality technology.

A further primary objective of the present invention is to provide a single-chip (SOC) system to effectively improve system construction costs, simplify control processes, and achieve system miniaturization.

A further major objective of the present invention is to provide a digital micro-module that is effective in integrating microscopic and processing technology-related devices to achieve modularization and systematization.

For the above purposes, the Augmented Reality Image Generation System provided by the present invention comprises a processing module and a digital micro-module of a plurality of camera units for capturing an instantaneous image of the object according to the control signal, and transmitting the image to the processing mode. The group, wherein the object system is bulk or mass suitable for microscopic objects that are processed through microscopic and convergent processing for observation and interaction. The processing module tracks or detects and analyzes the user's manipulation actions to generate a corresponding control signal, and receives a state including at least one object and/or at least one operation or feature region thereof according to the manipulation action or the control signal A momentary image, processing the instant image to generate at least one virtual object, and generating an augmented reality image of the superimposed virtual object. Wherein, if the manipulation action includes triggering an interactive application including opening or closing of the object display mode switching or instant guidance or sharing, the processing module is transparent, Instantiating or dynamizing a momentary image of a portion or all of the object and/or the state of the altered operation or feature region, and/or overlaying, invoking, and/or displaying an interface associated with the interactive application and the object, Augmented reality images before and after images, objects, videos, and/or information. Therefore, users can obtain relevant information through augmented reality technology, and because they can see the situation of operating virtual three-dimensional objects in the actual environment, they can produce immersive realism and good user experience. Can effectively improve the motivation of interaction, learning and use.

In the above embodiment of the present invention, the digital micro-module may have a convergence module, wherein the convergence controller unit and the mirror unit may be assembled, so that the convergence controller unit adjusts the mirror according to the control signal or an automatic adjustment rule. The unit and some of the photographic units capture the relative or geometric relationship of the instantaneous images to eliminate the blurring and related problems caused by insufficient convergence of the observer when viewing the fine objects.

In the above embodiment, the augmented reality image generation system may further include a light source module to mainly provide ambient illumination when the object is photographed, and the single-chip microcontroller interface module may actuate the digital micro-mode according to the control signal. The group and the display module can be a head-mounted display, a stereoscopic display or a flat-panel display to display augmented reality images, and the host computer or the portable electronic device and the display module can be configured as a near end, a far end or a cloud. The architecture controls and interacts. In other embodiments, the system can further include an operating platform and a positioning module for the digital micro-module to combine and at least one axially move in an operating space in response to the control signal. In this case, the manipulation action may be that the user operates in a real operation or an experimental actual instrument by a simulated operation object, a hand or an object, and is operated in and out of the operation space, or Approaching, touching, leaving, operating, inserting or securing some or all of the objects, and changing the state of the operation or feature area, or by user manipulation or coupling to the processing module The interface selects or applies the manipulation action. The user control interface module can be a pedal device, a manual joystick device, a hand-held or a head-mounted or wear-in interface device or a mobile communication device, and can also be combined with a digital microscope module. The operation parameter adjustment object and/or the display mode switching object is provided for the user to adjust the focal length, the zoom ratio, the moving distance, the rotation angle or the value of the light source parameter of the digital microscope module, and provide different and diverse user choices. Augmented reality image display or arrangement, such as single display, side-by-side display, and array display mode. In addition, in other embodiments, the processing module is further configured to perform image feature tracking, color detection, or motion detection on the instantaneous image to obtain a virtual object or determine whether to trigger or trigger an interactive application, and the evaluation module and The error warning module can generate or output an evaluation result, an unaligned response or a triggering operation prompt of the interactive application when the augmented reality image operated by the user meets or does not meet a preset specification. In addition, it can be combined with a learning feedback or community sharing module for users to store, edit, transmit or share augmented reality images, evaluation results, misaligned responses or triggering operational prompts.

For the above purposes, the microsurgery teaching or training system provided by the present application using an image penetrating technique is configured with at least the aforementioned augmented reality image generating system, or is configured to perform the operations as described above. A method in which an object organism is microscopic and true to a body or a tissue, or simulates the use of one specimen or a prosthetic model, such as an activity observation of an insect such as a butterfly, or imparts dynamism to a static specimen or historical artifact. Augmented reality image processing (such as overlaying a butterfly or fossil flapping flap or rotating animation or video options and content) to bring it to life, and capture the image of the transient image through digital microscopy The technology is processed by the processing module to generate virtual objects and superimposed and output to the display module, thereby achieving the technical effect of eliminating alignment errors and reducing delay.

In the above embodiment of the present invention, the object may be an ontology, tissue, specimen or prosthesis model of an eye, brain, skin or bone of an animal or a human, and if the object is an ontology, tissue, specimen of the eye Or a prosthetic model, and when the microsurgery includes a cataract, retina, macular or corneal surgery, the simulated operating object can be a probe device with a prompting mechanism, or the usual practice of the physician in the surgery or experiment. Instruments, such as capsular sacs, scissors or electric burners, can make training and practice closer, effectively increasing the experience of physician training and the ability to operate related surgery and research.

For the above purposes, the electronic component assembly training and detection system provided by the present application, which is an image penetration technology, is configured with at least the aforementioned augmented reality image generation system, or is implemented to implement the foregoing operation method. The object is a circuit board, a carrier or an electronic device for inserting or fixing an electronic component, and the image penetrating technology captures the object and/or at least by the digital microscope module. An instantaneous image of the state of an operation or feature area is processed by the processing module to generate a virtual object and superimposed and output to the display module to eliminate an alignment error and display delay. In the above embodiment of the present invention, the operating object is a probe device having a prompting mechanism, and the actual instrument for surgery or experiment includes an actual tool or instrument such as a welding torch or a forceps.

For the above purposes, the object microscopic observation and interaction system of the image-transparent technology provided by the present invention is configured with at least the aforementioned augmented reality image generation system, or is implemented to implement the foregoing operation method. The object is selected from micro-organisms, plants, minerals, organic matter, inorganic substances, chemical elements or compounds suitable for micromanipulation, and the image penetrating technique captures the object and/or at least by the digital micro-module An instant image of the state of an operation or feature area for processing by the processing module to generate a virtual object and to simultaneously output the output to the display module to eliminate alignment errors and reduce delay.

In the above embodiment of the present invention, if the manipulation action includes triggering the interactive application, the processing module is further configured to generate an instant image including at least part or all of the transparent or materialized object and/or overlap, call, and / or display an augmented reality image associated with an interface, image, object, and/or information associated with the interactive application and object; if the manipulation action includes a triggering interactive application that switches the display mode, the display mode is one a single display mode, a side-by-side display mode, or an array display mode, and the processing module is further configured to generate a transparent or materialized portion according to a single display mode, a side-by-side display mode, or an array display mode selected by a user. All objects and/or augmented reality images that are superimposed and displayed with one interface, image, object, and/or information associated with the object to produce objects that are identical, identical, or identical Augmented reality images displayed or arranged simultaneously.

To achieve the above objectives, the present invention further provides a single-chip (SOC) system comprising at least a processing module for simulating a system as described above or implementing the foregoing method of operation.

For the purpose of the above, the present invention further provides a digital micro-module for coupling or electrically connecting to a processing module such as the aforementioned system or a computer host or a computer connected or electrically connected or coupled thereto or Portable electronic device, or to implement the foregoing method of operation. The digital micro-module includes at least a photographing unit for transmitting a transient image containing at least an object and/or a state of at least one of the operations or feature regions according to a control action or a control signal generated by the processing module to the processing mode The group, wherein the object system is bulk or mass suitable for microscopic objects that are processed through microscopic and convergent processing for observation and interaction.

In the above embodiment of the present invention, the digital micro-module further comprises: an aggregation module, a positioning module, a user manipulation interface module and a display module. At least a part of the convergence module is a light splitting component, so that the photographing units respectively obtain an object that penetrates and reflects the light splitting component and/or Or an instantaneous image of the state of the operation or feature area; or, further comprising: a light splitting component, the separation component is disposed between the convergence module and the plurality of photography units, so that the photography unit respectively obtains the reflection module to reflect and penetrate A momentary image of the state of the object and/or the operation or feature area reflected by the beam splitting element.

1‧‧‧Augmented Reality Image Generation System

11‧‧‧Computer host or portable electronic device

121~125‧‧‧ objects

13‧‧‧ Positioning Module

131‧‧‧Operation platform

132‧‧‧Digital Micro Modules

133‧‧‧X-axis

14‧‧‧Single Chip Microcontroller Interface Module

151, 152‧‧‧ user control interface module

16‧‧‧Network

171, 172‧‧‧ display module

181‧‧‧simulated operating objects

182‧‧‧Hands

191, 192, 1003‧‧‧ augmented reality images

22‧‧‧Display mode switching object

231, 232‧‧ ‧ photography unit

24‧‧‧Light source module

241‧‧‧LED

31‧‧‧ Convergence module

311‧‧‧Mirror unit

400‧‧‧User eyes

411‧‧‧ first side of the mirror unit

412‧‧‧ second side of the mirror unit

421‧‧‧Light or video signal

422, 424~426‧‧‧Light or image

423‧‧‧Virtual objects

61~64‧‧‧Operating parameter adjustment object

70‧‧‧Manipulation actions or instruction analysis

71‧‧‧Digital module focal length, moving distance or rotation angle adjustment

72‧‧‧Light source parameter adjustment

73‧‧‧Magnification ratio adjustment

74‧‧‧Feature tracking

75‧‧‧Color detection

76‧‧‧ motion detection

77‧‧‧Interactive applications

78‧‧‧Evaluation module

79‧‧‧Error warning module

81‧‧‧ menu

821~823‧‧‧Operation or feature area

T1~T4‧‧‧ time point or interval

922~924, 1002‧‧‧ operation or feature area

1001‧‧‧ False eyes

1004‧‧‧Tack cap

1 is a block diagram of a system function of an embodiment of an augmented reality image generation system according to the present invention.

2 is a schematic diagram of the principle of microscopic image convergence according to an embodiment of the augmented reality image generation system of the present invention.

FIG. 3 is a structural diagram of a digital micro-module and a light source module according to an embodiment of the present invention.

4A is a structural diagram of a digital micro-module, a positioning module, and a light source module according to an embodiment of the present invention.

4B and 4C are respectively a partial component architecture and a spectroscopic operation diagram of different embodiments of the mirror unit of the augmented reality image generation system according to the present invention.

4D~4F are schematic diagrams showing part of the component structure and the optical splitting operation according to different embodiments of the convergence module and the light splitting component of the digital micro-module.

FIG. 5 is a structural diagram of a digital micro-module, a convergence module, a positioning module, and a light source module according to an embodiment of the present invention.

6A and FIG. 6B are respectively a structural diagram of a user manipulation interface and a simulated operation object according to an embodiment of the augmented reality image generation system according to an embodiment of the present invention.

7 is a processing mode of an embodiment of an augmented reality image generation system according to the present invention. Group function execution flow chart.

8A-8D are schematic diagrams of augmented reality images of different embodiments of a teaching or training system for microsurgery using an image penetrating technique according to the present invention.

9A and 9B are schematic diagrams showing augmented reality images of different embodiments of an electronic component assembly training and detecting system using an image penetrating technique according to the present invention.

10A and 10B are schematic diagrams showing augmented reality images of different embodiments of an object microscopic observation and interaction system using an image penetrating technique according to the present invention.

Please refer to FIG. 1, which is a system function block diagram of an embodiment of the augmented reality image generation system according to the present invention. As shown in FIG. 1 , in this embodiment, the augmented reality image generating system 1 has a processing module (not shown) or a positioning module that is coupled or coupled to the host computer or the portable electronic device 11 . The group 13, the operation platform 131, the digital micro-module 132, the single-chip microcontroller interface module 14, the user manipulation interface modules 151 and 152, and the display modules 171 and 172. The computer host or the portable electronic device 11 and the display module 171 can be electrically connected or configured as a near-end structure to display the augmented reality image 191, and can also be electrically connected to the display module 172 through the network 16 . Or configured as a remote or cloud architecture to display the augmented reality image 192 for remote transmission, control, sharing, etc.; the display modules 171 and 172 can be head mounted displays, stereoscopic displays, or flat panel displays. For displaying augmented reality images or stereoscopic images, and the display module 172 also discloses that it can be combined with a computing terminal or a server host to perform remote control of the system of the present invention; a single-chip microcontroller interface The module 14 and the processing module, or the computer host or the portable electronic device 11 in which the processing module is built, may be integrated or coupled to each other, or may be coupled or coupled to the processing module and Between digital micro-modules 132, according to the host computer or portable The control signal sent by the electronic device 11 or the single-chip microcontroller interface module 14 activates the digital micro-module 132. The operating platform 131, the digital micro-module 132, and the positioning module 13 can be further assembled separately or together with the display module 171, wherein the operating platform 131 can provide a real body for the user to place micro-organisms suitable for micromanipulation. , organization, simulation specimen or prosthetic model, or a circuit board, carrier or electronic device for the user to insert or fix electronic components, as an object to be observed, to be operated 121, here a butterfly The positioning module 13 of the bracket can be assembled by the solid and heavy-duty materials for the digital micro-module 132, and can be sent according to the computer host or the portable electronic device 11 or the single-chip microcontroller interface module 14. The control signal drives the MEMS and the motor control mechanism to enable the digital micro-module 132 to be in the operating space defined by the operating platform 131 (ie, to operate the upper surface of the operating platform 131 as an origin, for example, to a three-dimensional right angle coordinate At least one axial direction (for example, the X-axis 133) of the X-axis 133, the Y-axis, and the Z-axis extending in the axial direction is bidirectionally moved and positioned.

Please refer to Figure 1 again and refer to Figures 2~4C. 2 is a schematic diagram of a microscopic image convergence principle according to an embodiment of the present invention, and FIG. 3 is a digital micromodule and a light source module according to an embodiment of the present invention. FIG. 4A is a structural diagram of a digital micro-module, a positioning module, and a light source module according to an embodiment of the augmented reality image generating system of the present invention, and FIGS. 4B and 4C are respectively according to the present invention. A part of the component structure and the spectroscopic operation diagram of different embodiments of the mirror unit of the augmented reality image generation system are created. As shown in Fig. 2, the left side of Fig. 2 shows the interpupillary distance problem and phenomenon that will occur when imaging the left and right eyes to observe micro-organisms such as ants, and can be adjusted unilaterally or bilaterally by gradual or automatic rules of convergence processing. The camera captures the line of sight and eventually forms what is shown in the right diagram of Figure 2, thereby eliminating the aforementioned blurring and discrepancies. Therefore, as shown in Figures 1, 3 and 4A In an embodiment, the digital micro-module 132 is assembled to include at least two photographic units 231 and 232 for volume or mass selection according to the control signal for microscopic and convergent processing for observation and interaction. The state of the micro-object and/or the operation or feature area on the object (eg, due to changes in the interactive image content generated by the interactive application, this will be left to the subsequent paragraphs of the implementation and will not be described here) The image is transmitted to the processing module, and may further include a convergence module 31 to implement the foregoing aggregation operation.

The embodiment is continued, and please refer to FIG. 5, which is a digital micro-module, a convergence module, a positioning module, and a light source module according to an embodiment of the present invention. Architecture diagram. The convergence module 31 of the embodiment shown in FIG. 4A is assembled and has a convergence controller unit (which has been integrated, is configured as a control circuit and its controlled pivoting component, and therefore is not directly labeled) and has a reflection. The mirror unit 311 (formed here as a V-shaped surface, each side facing the reflected light can be designed or assembled according to the different light source requirements and applications), wherein the convergence controller unit is adjusted according to the control signal The mirror unit 311 and the photographing units 231 and 232 capture the relative or geometric relationship of the instantaneous images to achieve the result of the converging operation. The light source module 24 is annularly assembled by, for example, the LEDs 241 and is hollowed out at the center for light reflection or instant maintenance. In the embodiment, the light source module 24 is assembled to provide a projection to the center point of the object and is reflected to the mirror unit 311 (in this case, a V-shape, and The back side is a mirror surface that splits and refracts light to the photographing units 231 and 232, which substantially does not create shadows or shadows on the object. In addition, in the embodiment shown in FIG. 4B and FIG. 4C, the first surface 411 and the second surface 412 of the mirror unit 311 can be respectively assembled or coated as a mirror surface, the difference being that the light or image signal 421 passes through different paths or distances. The reflected, refracted or transmitted light or images 422, 424 may result in a blur or sharpness of the captured instant image being viewed by the user's eyes 400.

As shown in FIG. 6A, FIG. 6B, FIG. 6A and FIG. 6B are respectively a user manipulation interface and a simulated operation object according to an embodiment of the augmented reality image generation system according to an embodiment of the present invention. The composition diagram is composed, and FIG. 7 is a flowchart of executing the function of the processing module according to an embodiment of the augmented reality image generation system according to the present invention. In the foregoing and the present embodiment, the processing module can be configured to track, detect, and analyze 70 user-selected or coupled manipulation actions selected or applied by the user manipulation interface 151 of the processing module or Commands to generate corresponding control signals to, for example, the light source module 24 or the positioning module 133, and receive objects and/or operations that are captured and transmitted by the digital microscope module 132 in response to the manipulation or control signals. The instantaneous image of the state of the feature area or the augmented reality images 191 and 192 of the virtual object generated by the superimposition are sent to, for example, the display module 171 for display. The manipulation action is performed by the user by simulating the operation object 181, the hand 182 or the object of the surgical or experimental actual instrument (not shown) into or out of the operation space, and/or approaching, contacting, leaving, and operating. Inserting or securing some or all of the objects and/or changing the state of at least one of the operations or feature areas. If the manipulation action includes triggering an interactive application 77, and wherein the interactive application 77 includes at least switching of the display mode of the object or opening and closing of the instant guidance or sharing, the processing module is further configured to generate a transparent or materialized portion or Instantaneous image of all objects and/or states of at least one operation or feature area after the change, and/or overlay, call and/or display of an interface, image, object associated with the interactive application 77 and the object And/or augmented reality images after the information. The processing module can be further configured to perform image feature tracking 74, color detection 75 or motion detection 76 on the instantaneous image captured by the digital micro-module 132 to obtain a virtual object or to judge the instantaneous image. To decide whether to trigger or have triggered the interactive app. Alternatively, the processing module may further include an evaluation module 78 and/or an error warning module 79, which are configured to match whether the augmented reality image generated by the processing module meets or does not meet a preset specification. Generate or lose accordingly Trigger operation prompts for evaluation results, misalignment reactions, or interactive applications, such as sound, light, voice, or video display. The processing module may further comprise a learning feedback or community sharing module (not shown), which is configured for the user to store, edit, transmit or share the augmented reality image, the evaluation result, the misalignment Respond or trigger an action prompt.

In the embodiment, the user manipulation interface module 151 can be a foot pedal device, or can be a manual joystick device, a handheld input/output interface device, or a mobile communication device. As shown in FIG. 6A, the user manipulation interface module 151 is further provided with the operation parameter adjustment objects 61-64 of the digital micro-module 132 and the display mode switching object 22. The operation parameter adjustment objects 61-64 are assembled for the user to adjust the focal length, the moving distance or the rotation angle 71 of the digital microscope module 132, the light source parameter 72, the zoom magnification 73, and the display mode switching object 22 The user is configured to select a single, multiple identical or different objects 122-125 to simultaneously display or arrange the augmented reality image, and the display mode is selected from a single display mode (shown in FIG. 6B). A side-by-side display mode (not shown in the figure, but similar to FIG. 8C, two of which are shown) and an array display mode (please refer to FIG. 8C first).

Please refer to FIG. 4D to FIG. 4F , which are schematic diagrams showing part of the component structure and the optical splitting operation according to different embodiments of the convergence module and the light splitting component of the digital micro-module. The placement positions of the photographing units 231 to 233 can be adjusted according to different convergence modules 31, mirror units 311, 312 depending on the application and requirements, or the coating is mirror-finished, and reflected by different paths or distances. , refracting and transmitting light or images 425, 426 and the convergence operation and principle shown in FIG. 2, the design and assembly of the system or module architecture for capturing the instantaneous image (in this embodiment, for example, an ant's miniature object) For the quality of subsequent augmented reality images, there will be blurred and clear quality differences.

Since the single-chip (SOC) system of the present invention includes at least one processing module to simulate the processing module of the system, it should be fully disclosed and explained by the foregoing embodiments and corresponding drawings and can be realized accordingly. , will not repeat them here.

Please refer to FIGS. 8A-8D, which are schematic diagrams of augmented reality images of different embodiments of an object microscopic observation and interaction system using an image penetrating technique according to the present invention. In this embodiment, the object microscopic observation and interaction system is configured with at least the augmented reality image generation system described above, or is configured to perform the method of operation as described above, and the object is selected from microscopic operations. Micro-organisms, plants, minerals, organic matter, inorganic substances, chemical elements or compounds, such as the object butterfly 122-125 in this embodiment, and the image penetrating technology captures the object and/or by the digital micro-module. Or a transient image of at least one of the operational or characteristic regions 821-823 for processing by the processing module to generate a virtual object, such as the menu symbol shown above the rightmost figure of FIG. 8A and the butterfly superimposed on the butterfly image. The larva virtual object, the menu of Figure 8B, the information, the array display mode, the snapshot, the image or the object shared to the community, can be synchronously output to the display module after being superimposed to eliminate an alignment error and display delay, and can be improved The use or viewer has a rich learning experience for the visualization or dynamism of the butterfly's ecological processes in this embodiment. In addition, as shown in FIG. 8C, after the instantaneous image of the state of the object 121 and the operation or feature area is captured, the information related to the object 121 in the array display mode (eg, similar or the same or different biological classification) The butterflies can be coexisted and displayed on the operating platform for observation and operation. The menus can also be superimposed to produce and display or temporarily hidden for subsequent call execution.

In addition, as shown in FIG. 8D, for example, a specimen 121 or a still object 121 (here, a butterfly) can be created according to the present invention to generate an augmented reality image; in detail, the system To capture the real-time image of the object as a texture material for 3D animation to make the viewer Producing the illusion that the real specimen or the still life is dynamic, so when the time axis is advanced from the time point or the interval T1 to T4, the animation or the movie of the state of the object 121 can be continuously presented, that is, the wing can be indicated as flapping wings or Flying near or away from the viewer to cause the appearance of images or animations such as dimensional changes, can also be indicated as advancing by time. The microscopic organisms to be microscopically observed can be of the same type or different types, so the creation department has multiple applications and Rich fun and cross-domain learning help.

Please refer to FIG. 9A and FIG. 9B , which are schematic diagrams of augmented reality images of different embodiments of an electronic component assembly training and detection system applying an image penetrating technique according to the present invention. In this embodiment, the electronic component assembly training and detection system is configured with at least the augmented reality image generation system, and the object is a circuit board, carrier or electronic device for the user to insert or fix the electronic component. The device, and the image penetrating technique captures a momentary image of the state of the object and/or at least one of the operations or feature regions by the digital microscope module for processing by the processing module to generate the virtual object and the synchronization after the overlay Output to the display module to eliminate an alignment error and display delay. As shown in FIG. 9A, the object is an IC here, and has a plurality of pin pins. Each pin can transmit and receive different input or control signal contents. Therefore, according to the present embodiment, the IC is related to the IC. Interfaces, images, information, and menus can be superimposed and displayed on a virtual reality image for users or viewers to operate or invoke without having to leave the system separately or frequently to seek the required reference information, thus being forced Interrupt observation and learning process.

In the embodiment shown in FIG. 9B, the object is a circuit board for inserting or fixing electronic components, and the digital micro-module captures an object and/or at least one operation or feature thereof. The instantaneous image of the state of the area 922~924, that is, the image of the board that captures the real environment, and the image of the solder joint/hole of the electronic component to be inserted or to be soldered in the operation or feature area 922~924 After that, all the virtual objects are not actually plugged in by the user. All of them will be presented before the device; then, as the user's manipulation starts, the resistors that are to be plugged into the board will be obtained according to the operator's control action from the time point or interval T1~T3. When the information such as the resistance value or color of a virtual object such as a capacitor or an IC is related to the augmented reality image, the operator's handheld IC, or the feature area 923, the virtual object in the area will automatically disappear, and the comparison is still visible at this time. The difference between the augmented reality image in which the object appears in the operation or feature area 924 and the augmented reality image containing the actual and virtual objects after actual assembly can be seen. In other embodiments of the present invention, the system is still operable to prevent the operator from viewing the blocked or restricted or implemented interference, and the processing module detects the operator's hand portion or gradually An augmented reality image different from T2 in FIG. 9B may be generated when an action trend in the captured image or operation or feature area is captured or captured, ie, for example, the handheld IC proximity operation or feature area 923 is generated. The state of the time changes before or after, or other conditions, as a temporary prohibition mechanism, temporarily remove, transparent or disable the instant guidance or sharing of the opening and closing and the corresponding augmented reality image generated, or Corresponding to generate the augmented reality image without superimposing the virtual object to avoid interference with the user's operation, that is, removing or transparent, disabling or turning off some instant guidance or sharing opening and closing functions and related interaction guides The interface and the information content, so that the operator is not disturbed, and can still be based on the evaluation or warning or sharing modules and mechanisms provided in other embodiments of the present creation, thereby not only satisfying the alignment error Delay of demand, will receive more benefits caused by custom and application of information technology-assisted learning in business and industry.

Please refer to FIG. 10A and FIG. 10B , which are schematic diagrams of augmented reality images of different embodiments of the teaching or training system for microsurgery according to the present application. In this embodiment, the teaching or training system for microsurgery is configured with at least the aforementioned augmented reality image generation system, or is configured to perform the aforementioned operation method, the object organism A miniature or real body or a tissue, or a model or a prosthetic model. In this embodiment, the object is an ontology, tissue, specimen or prosthetic model of the eye, brain, skin or bone of an animal or human, and is an ontology, tissue, specimen or prosthetic model of the eye, Microsurgery includes, but is not limited to, cataracts, retina, macula, or corneal surgery. In this embodiment, for example, having a false eye 1001, the processing module can be used as a cataract surgery according to at least the state of the object false eye 1001 and the operation or feature area 1002 (ie, the border of the eyeball plus the marking block above the eye). The capsular tag or the virtual object 1003 that is guided immediately, the operator can utilize the probe device with the prompt mechanism or the actual instrument for surgery or experiment, such as the capsular sac 1004 or scissors, the electric burner, etc., with the system of the present invention. The teaching or training system of the capsulotomy is performed according to the augmented reality image generated by the superposition.

The above-mentioned embodiments are only examples, and are not intended to limit the scope of the present invention; any simple or equivalent changes and modifications made without departing from the spirit and scope of the present invention should still fall within the scope of the present invention.

1‧‧‧Augmented Reality Image Generation System

11‧‧‧Computer host or portable electronic device

121‧‧‧ Objects

13‧‧‧ Positioning Module

131‧‧‧Operation platform

132‧‧‧Digital Micro Modules

133‧‧‧X-axis

14‧‧‧Single Chip Microcontroller Interface Module

151, 152‧‧‧ user control interface module

16‧‧‧Network

171, 172‧‧‧ display module

191, 192‧‧Augmented Reality Image

Claims (20)

An augmented reality image generation system includes: a processing module configured to track or detect and analyze a user's manipulation action to generate a corresponding one of the control signals, receive the response action or The control signal captures a transient image including at least one object and/or one of at least one operation or feature region, processes the transient image to generate at least one virtual object, and superimposes the virtual object to augment the reality An image; wherein, if the manipulation includes triggering an interactive application, and wherein the interactive application includes at least switching of a display mode of the object or opening and closing of an instant guide or sharing, the processing module is further configured to generate Transmitting, embodying, or dynamizing the partial image of the object and/or the state of at least one of the operations or feature regions after the change, and/or superimposing, invoking, and/or displaying the interaction with the interaction Applying the augmented reality image before and after one of the interfaces, images, objects, videos, and/or information associated with the object; and a digital microscopic module that is assembled And comprising a plurality of photographing units for extracting the instant image according to the control signal and transmitting the image to the processing module, wherein the object system is volume or mass suitable for microscopic and convergent processing for observation and interaction Micro objects. The augmented reality image generation system of the first application of the patent scope, the digital micro-module further comprises: a convergence module, which is further configured to include a convergence controller unit and a mirror unit, wherein The convergence controller unit adjusts the relative or geometric relationship between the mirror unit and the camera unit to capture the instantaneous image according to the control signal or an automatic adjustment rule. The augmented reality image generation system of claim 2, further comprising: a light source module configured to provide a projection to the center point of the object and reflect to the opposite The mirror unit splits and refracts light to one of the photographing units, wherein the light does not substantially create a shadow or shadow on the object. The augmented reality image generation system of claim 1 further includes: a single chip microcontroller interface module, which is assembled or coupled to the processing module or the processing module and the digit Between the micro-modules, the digital micro-module is actuated according to the control signal. The augmented reality image generation system of claim 1, further comprising: a display module configured to be a head mounted display, a stereoscopic display or a flat display to display the augmented reality image, and Electrically connected or configured in the processing module to be coupled to or coupled to a computer host or a portable electronic device, wherein the computer host or the portable electronic device and the display module are assembled as a proximal end , remote or cloud architecture. The augmented reality image generation system of claim 5, further comprising: an operation platform configured to be used by the user to place the object; and a positioning module configured to provide The digital micro-module is combined and adapted to move in at least one axial direction of one of the operating spaces defined by the operating platform in response to the control signal; wherein the manipulation is performed by the user by a simulated operation object , one hand or one of the objects, surgical or experimental actual instrument entering or removing the operating space, and/or approaching, contacting, leaving, operating, inserting or securing some or all of the object and/or changing at least The state of the operation or feature area. The augmented reality image generation system of claim 1, further comprising: a user manipulation interface, configured or coupled to the processing module, for the user to select or apply the manipulation action. For example, the augmented reality image generation system of claim 7 of the patent scope, the user manipulation interface module a foot pedal device, a manual joystick device, a hand-held or head-mounted or wear-in interface device or a mobile communication device, and further equipped with one of the digital microscope modules operating parameter adjustment items and/or A display mode switches objects. The augmented reality image generation system of claim 8 is configured to adjust the focal length, zoom magnification The value of the parameter. The augmented reality image generation system of claim 7, wherein the display mode switching object is configured to allow the user to select a single, plural, or different object to simultaneously display or arrange the amplification. The real image, and the display mode is selected from the group consisting of a single display mode, a side-by-side display mode, and an array display mode. The augmented reality image generation system of claim 1 further includes: an evaluation module and/or an error warning module, which is assembled to generate the amplification generated by the processing module. When the image conforms to or does not meet a preset specification, an evaluation result, an unaligned response, or one of the interactive applications is triggered to generate an operation prompt. For example, the augmented reality image generation system of claim 11 further includes: a learning feedback or community sharing module, which is configured to store, edit, transmit or share the augmented reality. Image, the result of the evaluation, the misalignment reaction, or the triggering operation prompt. An augmented reality image generation system according to any one of claims 1 to 12, wherein the object system is at least one of the teaching or training systems for microsurgery using an image penetrating technique. The organism is microscopic and real one body or a tissue, or a model or a prosthetic model. An electronic component assembly training and detection system using an image penetration technology, at least configured The augmented reality image generation system according to any one of claims 1 to 12, wherein the object is for the user to insert or fix a circuit board and a carrier of an electronic component. Or an electronic device. An object microscopic observation and interaction system using an image penetrating technique, which is provided with at least the augmented reality image generation system according to any one of claims 1 to 12, wherein the object selection system From micro-organisms, plants, minerals, organics, inorganics, chemical elements or compounds suitable for micromanipulation. A single-chip (SOC) system is provided with at least the processing module of the augmented reality image generation system according to any one of claims 1 to 12. A digital micro-module for coupling or electrically connecting the processing module of the augmented reality image generation system as described in claims 1 to 12, or a combination thereof The computer host or the portable electronic device that is connected or coupled to at least the camera unit includes at least the object and the control signal generated by the processing module according to the control action And/or the instantaneous image of the state of at least one of the operational or feature regions is transmitted to the processing module, wherein the object is bulk or mass-sized for microscopic objects that are microscopically and convergently processed for observation and interaction . The digital micro-module of claim 17 of the patent application, further comprising the convergence module, the positioning module, and the user of the augmented reality image generation system according to claim 1 to 12 Manipulating the interface module and/or the display module. For example, in the digital micro-module of claim 18, at least a part of the convergence module is a light-splitting element, for each of the imaging units to obtain the object that penetrates and reflects the light-splitting element and/or The momentary image of the state of the operation or feature area. The digital micro-module of claim 18, further comprising: a light splitting component, the separation component being disposed between the convergence module and the camera units, wherein the camera units respectively obtain the convergence module The transient image that is reflected and then reflected by the object of the spectroscopic element and/or the state of the operation or feature region.