JP5841310B2 - System, apparatus, method and computer program product for determining spatial characteristics of an object using a camera and a search pattern - Google Patents

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application is a U.S. patent application filed on June 11, 2008. 12 / 137,338, a continuation-in-part application, the contents of which are incorporated by reference into the entire application.

BACKGROUND OF THE INVENTION Field of the Invention The present invention relates generally to optical position recognition, and more particularly to multiple measurements of an object used in acquiring a three-dimensional image of the object (ie, acquired images, eg, images, scanned images, etc. The present invention relates to a method, a system, an apparatus, and a computer program for using an optical position recognition description for correlating frame data acquired during ().

Related technology

In a conventional three-dimensional measurement system with a narrow field of view, used to acquire images of relatively large objects, such as actual or dentures, or tooth molds or tooth castings, the measurement field of the optical meter Alternatively, the measurement amount is smaller than the measurement amount of the object to be measured. Therefore, in order to obtain a substantially complete image of the object, it is necessary to perform multiple measurements at various portions of the object. Between measurements, the object moves relative to the optical measurement system. The data obtained from each measurement needs to be correlated, that is, mapped to a common coordinate system, in order to obtain a composite 3D image for the entire object.

  A conventional three-dimensional measurement system may use a mechanical association technique in order to associate data obtained in multiple measurements. FIG. 10A shows an example system 1000 using a conventional mechanical association technique for associating three-dimensional data obtained in multiple measurements. The system 1000 includes a measurement optical system 1002 and a slide 1004. The support member 1006 positions the measurement optical system 1002 relative to the slide 1004 in the fixing direction. Thereby, the optical measurement system 1002 and the slide 1004 do not move relatively. The mechanical grid 1008 is provided on the upper surface of the slide 1004. The object 1010 is fixed to the object holder 1012. The object holder 1012 is installed at a predetermined position on the mechanical grid 1008. At each position, the measurement is performed once, and one frame of three-dimensional data is acquired. By combining the frame data according to a known frame association technique, a combined three-dimensional image of the entire object is created. The disadvantage of this system 1000 is that although the object holder 1012 can only be set at a predetermined position allowed by the grid 1008, this location is not necessarily the optimal position for acquiring 3D data. .

  The conventional three-dimensional measurement system may use an optical correspondence technique in order to associate frame data obtained by a plurality of measurements. The position is determined by the position of the reference object placed on the object holder. For example, Cercon Eye Scanner of DeguDent GmbH adopts optical matching technology.

  FIG. 10B shows an example system 1050 that uses conventional optical alignment techniques to correlate 3D data acquired in multiple measurements. The system 1050 includes a measurement optical system 1052 and a slide 1054. The support member 1056 is installed in a direction in which the measurement optical system 1052 is relatively fixed to the slide 1054 so that no relative movement occurs between the measurement optical system 1052 and the slide 1054. The object 1058 is fixed to the object holder 1060. The object holder 1060 includes a reference position marker adjuster 1061 that positions the reference position marker 1062 on the object 1058. The object holder 1060 then moves across the slide 1054 in discontinuous steps. Measurement and acquisition of a frame of three-dimensional data are performed in each step. Each measurement needs to include a reference position marker 1062. Optical matching techniques are used to identify the reference position marker 1062 and create corresponding position information for each frame of the three-dimensional data. By combining the frame data according to a known frame association technique, a combined three-dimensional elephant is formed in the entire area of the object.

  Measurement optics 1052 typically has a camera (not shown) that is used to observe reference position marker 1062 on object holder 1060. The disadvantage of the system 1050 is that the reference position marker 1062 needs to be in view during each measurement. While the measurement is taking place, the reference position marker 1062 must not be covered by the object 1058 and must be visible by the camera.

  To calibrate the measurement optics of a photogrammetry system such as the system shown in FIGS. 10A and 10B, a calibration pattern having a circular structure is used. These calibration patterns are typically composed of unfilled circular structures having a constant radius, which are placed on a rectangular grid. Conventionally, the entire calibration pattern is set within the field of view of the measurement optical system to be configured. Since the entire area of the calibration pattern is used to determine the measurement index of the measurement optical system, it is not necessary to determine the coordinate position of the calibration pattern based on the partial field of view of the calibration pattern.

  The calibration pattern described above may be formed by offset printing on a ceramic substrate. However, since the reproducibility of the offset pattern is on the order of several tens of microns, the resulting calibration pattern is insufficient for the calibration process. Each such calibration pattern can be calibrated by separate measurements and a lookup table created for each calibration pattern. Each lookup table includes the exact center position of each circular structure. When using a look-up table, calibration of the measurement optical system using such a calibration pattern requires all calibration patterns in the field of view of the measurement optical system. Therefore, in such an application, the feature amount of the calibration pattern is not encoded.

  Other methods may be used to form such a calibration pattern. For example, a process similar to a semiconductor lithography process for creating semiconductor devices may be used to form a calibration pattern. In this process, a mask image may be drawn on a photoresist stacked on a thin chrome layer mounted on a quartz substrate. This substrate is then irradiated with light, and areas not irradiated with light in the next etching step (ie, masked areas) are removed. In other lithographic processes, for example, a light emulsion may be applied to the mylar foil. If such a lithography process is used, an error of 2 to 3 microns can be suppressed over the whole area of 1 inch. Therefore, by using such a lithographic process, it is possible to form a highly accurate calibration pattern.

  As noted above, when the measurement optics is calibrated using a conventional calibration process, the measurement optics will obtain images of all calibration patterns. Thus, the conventional calibration process does not use a position matching technique to determine the position of the calibration pattern using the acquired image data corresponding to a portion of the calibration pattern. However, a position recognition technique that uses partial information of a search pattern is known. For example, position recognition techniques may be used in conjunction with two-dimensional barcodes coded according to the Data Matrix standard (ISO / ICE16022 international code specification). Two-dimensional barcodes are created by coding information with a high degree of redundancy. For this two-dimensional barcode, a partial image may be acquired, and in this case, an error correction algorithm may be used to determine a position corresponding to the acquired image. However, two-dimensional barcodes are not designed to extract unknown information initially. That is, when only information about a partial barcode is known, the absolute coordinates of the two-dimensional barcode cannot be determined.

  The present invention overcomes the above limitations in measuring a three-dimensional object using conventional frame matching techniques.

The present invention meets the above recognized needs by providing a method, system, apparatus, and computer program product for evaluating an object placed on the top surface of an object holder. According to various embodiments of the present invention, measurement data can be obtained with superiority in multiple measurements of an object. These measurement data are associated with each other and are combined to create composite measurement data of the object.

  In one exemplary aspect of the present invention, there is a method for evaluating an object placed on an upper surface of an object holder. At least one first frame representing the acquired portion of the object is acquired while positioning the object holder at each of the plurality of positions. Acquiring at least one second frame representing the acquired portion of at least one other surface of the object holder while positioning the object holder at each of the plurality of positions. At least one spatial characteristic associated with the acquired portion of the object is determined based on at least one of the acquired frames. The values of the plurality of optical markers acquired in each of the second frames (at least two of the optical markers have different values) are determined. Determine at least one coordinate associated with the value and the orientation of the acquired portion on at least one other surface.

  The method may also include a step of creating composite data based on the acquired frame. This synthesized data may form a three-dimensional image of the object image. The method may further include a step of converting, for each first frame, coordinates related to the acquisition portion of the object based on the orientation and the coordinates. The determination process for determining at least one of the coordinates and the orientation is a process of determining the coordinates of the central portion of each optical marker of the plurality of optical markers in the first coordinate system, and the correspondence between the plurality of optical markers. Generating a value associated with one central portion, and obtaining a coordinate associated with one or more of the plurality of feature values in the second coordinate system. .

  Each of the plurality of optical markers may have a circular shape, and each feature value may be a corresponding radius of the plurality of circular optical markers. The plurality of optical markers may form an array. Data indicating the spatial characteristics may be stored and saved, or may be transmitted as a message. Each feature value may indicate at least the color of the corresponding optical marker or the distance between at least two optical markers. The characteristic may include at least one of size, shape, and color.

  In another embodiment of the present invention, there is provided an apparatus, system, and computer program product that operate according to the above method.

  In another embodiment of the present invention, a method for creating an optical pattern is provided. In the first coordinate system, the coordinates of a plurality of predetermined positions on the substrate are determined. An array of feature values corresponding to the plurality of predetermined positions on the substrate is generated. A plurality of optical feature structures are provided at locations on the substrate associated with the feature values. The plurality of optical feature structures may be provided on the substrate using a precision stamping process, a lithography process, and / or a printing process.

  In yet another example embodiment of the present invention, a system for evaluating an optical coding pattern is provided. An image acquisition unit is arranged to acquire at least one frame of image data corresponding to the acquired portion of the optical coding pattern including an array of optical feature structures, the frame of image data including an image of the optical feature structure. It is out. A processor associates the array of optical feature structures with an array of feature values corresponding thereto, and the acquisition portion of the optical coding pattern based on at least one of the values and images of the array of optical feature structures; Operative to determine at least one spatial characteristic associated with the.

  The at least one spatial characteristic may include at least one of coordinates and orientations of at least a part of the optical coding pattern. The processor may also operate to determine coordinates corresponding to the center of each optical feature structure. In addition, the system includes an object holder where the optical encoding pattern is disposed in relation to a lower surface thereof, and at least one measurement frame acquired by the object disposed on the upper surface of the object holder. May include a measuring unit arranged to obtain In this case, the processor further functions to generate combined measurement data based on at least one measurement frame acquired by the measurement unit and at least one frame of the image data acquired by the image acquisition unit.

  The features and advantages of the present invention will become more apparent from the following detailed description when taken in conjunction with the drawings. It should be noted that like reference numbers indicate identical or functionally similar parts.

FIG. 1A shows a system in an exemplary embodiment of the invention. FIG. 1B shows the measurement range of the system shown in FIG. 1A. FIG. 1C shows an example of the optical coding pattern shown in FIG. 1A. FIG. 1D shows in perspective view the components of the system shown in FIG. 1A. FIG. 1E shows a portion of the optical coding pattern shown in FIG. 1C and the field of view of the camera portion of the system shown in FIG. 1D. FIG. 2 illustrates an exemplary process for acquiring and relating three-dimensional data using the system shown in FIG. 1A. FIG. 3A shows a first example of multiple measurements of an object using the system shown in FIG. 1A. FIG. 3B shows a first example of multiple measurements of an object using the system shown in FIG. 1A. FIG. 3C shows a first example of multiple measurements of an object using the system shown in FIG. 1A. FIG. 3D shows a first example of multiple measurements of an object using the system shown in FIG. 1A. FIG. 3E shows a first example of multiple measurements of an object using the system shown in FIG. 1A. FIG. 3F shows a first example of multiple measurements of an object using the system shown in FIG. 1A. FIG. 3G shows a first example of multiple measurements of an object using the system shown in FIG. 1A. FIG. 3H shows a first example of multiple measurements of an object using the system shown in FIG. 1A. FIG. 3I shows a first example of multiple measurements of an object using the system shown in FIG. 1A. FIG. 3J shows a first example of multiple measurements of an object using the system shown in FIG. 1A. FIG. 3K shows a first example of multiple measurements of an object using the system shown in FIG. 1A. FIG. 3L shows a first example of multiple measurements of an object using the system shown in FIG. 1A. FIG. 3M shows a first example of multiple measurements of an object using the system shown in FIG. 1A. 4A to 4D show a second example of multiple measurements of an object using the system shown in FIG. 1A. FIGS. 5A-5D show the optical coding patterns shown in FIG. C, respectively, obtained from the camera of the system shown in FIG. 1A during the respective measurements shown in FIGS. 4A-4D. FIG. 6A to 6D show the coordinates in the coordinate system of the optical coding pattern shown in FIG. 1C obtained as a result of the measurement of the object of FIGS. 4A to 4D. 7A to 7D show the transformation of coordinates from the coordinate system shown in FIGS. 6A to 6D to the reference coordinate system. FIG. 8A shows the transformation from the coordinates of the data obtained in each measurement shown in FIGS. 4A to 4D to the reference coordinate system shown in FIGS. 7A to 6D. FIG. 8B shows the transformation from the coordinates of the data obtained in each measurement shown in FIGS. 4A to 4D to the reference coordinate system shown in FIGS. 7A to 6D. FIG. 8C shows the transformation from the coordinates of the data obtained in each measurement shown in FIGS. 4A to 4D to the reference coordinate system shown in FIGS. 7A to 6D. FIG. 8D shows the transformation from the coordinates of the data obtained in each measurement shown in FIGS. 4A to 4D to the reference coordinate system shown in FIGS. 7A to 6D. FIG. 8E shows the combined measurement data resulting from the measurement data shown in FIGS. 8A-8D. FIG. 9 shows a block diagram of the system structure of the system used with the system shown in FIG. 1A, according to an exemplary embodiment in accordance with the present invention. 10A and 10B show a conventional three-dimensional measurement system. FIG. 11 shows an exemplary optical coding pattern according to another embodiment of the present invention. FIG. 12 shows an exemplary process for creating the optical coding pattern shown in FIG. FIG. 13 illustrates a typical example of a process for determining the spatial characteristics of an optical coding pattern, according to one embodiment of the present invention. FIG. 14 shows further detailed examples of techniques that can be performed in step S1314 of FIG. 13 according to one exemplary embodiment of the present invention. FIG. 15A illustrates an example of a table of characteristic values corresponding to an optical feature structure, according to one embodiment of the present invention. FIG. 15B shows a portion of the master array of typical characteristic values shown in FIG. FIG. 15C shows a portion of the master array of typical characteristic values shown in FIG. 15B and a portion of a typical optical coding pattern encoded using the optical feature structure shown in FIG. 15A. Indicates. FIG. 15D illustrates a typical camera field of view according to an example of the present invention. FIG. 15E shows a portion of the optical coding pattern described in FIG. 15C and the field of view of the camera shown in FIG. 15D. FIG. 15F shows an exemplary sub-array of characteristic values corresponding to feature structures in the camera field of view shown in FIG. 15E. FIG. 16A shows a sub-array of characteristic values shown in FIG. 15E that is located in the master array of characteristic values described in FIG. FIG. 16B shows the coordinates corresponding to the characteristic value sub-array located in the characteristic value master array described in FIG. 16A. FIG. 17A shows a camera coordinate system with an identifier located at the center of each feature in the field of view of the camera shown in FIG. 15E. FIG. 17B shows the coordinates of the center of the feature structure shown in 17A and the coordinate system of the camera. 18 shows a reference coordinate system of a typical optical coding pattern that includes an identifier of the center position of the feature structure in the camera field of view shown in FIG. 15E, and the camera field of view shown in FIG. 15E. Indicates the identifier located at the center. FIG. 19 shows, by way of example, a typical process for acquiring and correlating three-dimensional data using other optical coding patterns as shown in FIG. Detailed Description Outline

Exemplary embodiments of the present invention relate to methods, systems, apparatus, computer programs that use optical recognition techniques to correlate measurement data acquired in multiple measurements of the surface of an object, even if Even when the field of view of the measurement unit is smaller than the size of the object, it is used to obtain a composite image of the object. Although this application describes a dental device that obtains a three-dimensional image of an actual or denture, or tooth mold or tooth casting, each example obtains a composite image of any type of object Also useful.

2. system

  The following description describes an exemplary system in which an exemplary embodiment of the invention is implemented. This is for illustrative purposes only and is not intended to limit the scope of application of the present invention to only examples. It will be apparent from this description to those skilled in the relevant art how the present invention may be implemented in other alternative embodiments.

FIG. 1A shows a system 100 in an exemplary embodiment of the invention. The system 100 makes it possible to acquire an image that is used to form a three-dimensional image of an object 102, such as a tooth or other object. The system 100 includes a measurement unit 104 and a slide 106. The support member 108 positions the measurement unit 104 in a fixed direction relative to the slide 106, whereby the measurement unit 104 and the slide 106 do not move relative to each other.

The measuring unit 104 may include any conventional suitable or three-dimensional measuring device that will be developed in the future. For example, the measurement unit 104 may include a fringe projection system including a fringe projector and a camera. However, the measurement unit 104 used in the present invention is not limited to the fringe projection unit. Other three-dimensional measurement devices may be used, for example, the measurement unit 104 may use confocal laser microscopy, optical coherence / tomography, white light interferometry, or other techniques known in the art. May be. For example, US Pat. No. 4,837,732 (Brandestini et al.), 4th tier, line 51 to 7th tier, 61th, and US Pat. No. 6,885,464 (Pfeiffer et al.) 4th tier, 15th line. 14th line and 48th line disclose a system suitable for use in the measurement unit 104. These patents are incorporated by reference as if fully described throughout the project.

The object to be measured 102 is attached or fixed to an object holder 110, and the object holder 110 has an optical encoding pattern 112 attached to the lower part thereof. Of course, the optical encoding pattern 112 may be integrated with the lower part of the object holder 110 without leaving the microscope of the present invention, and this does not depart from the scope of the present invention. The camera unit 114 is arranged inside the slide 106, or in another embodiment, below the slide 106 so that the optical encoding pattern 112 is within the field of view of the camera unit 114. A transparent portion 116 is provided on the upper surface of the slide 106. The transparent portion 116 allows the camera unit 114 to visually recognize at least a part of the optical encoding pattern 112 and acquire image data representing the portion. This data is subjected to data processing to obtain the spatial characteristics of the optical encoding pattern 112, for example, the orientation and relative position of the optical encoding pattern 112 with respect to the camera unit 114. As will be described later, the spatial characteristics of the optical coding pattern 112 are used to measure the spatial characteristics of the corresponding object holder 110 such as the orientation and relative position of the object holder 110 with respect to the measurement unit 104.

FIG. 1B illustrates the object 102 and the object holder 110 shown in FIG. 1A from a perspective that perspectively looks down on these components. The object 102 includes an upper right part A, a lower right part B, a lower left part C, and an upper left part D. The measurement unit 104 acquires measurement data from the measurement field 118 of the measurement unit 104. In this case, the term “measurement field of view” may be rephrased as “view”, “measurement range”, and “measurement amount”. Since the measurement field 118 may be smaller than the object 102, multiple measurements are performed according to one aspect of the present invention to obtain measurement data from the object 102 that is used to obtain a three-dimensional image thereof. There is also a thing.

FIG. 1C illustrates an exemplary optical coding pattern 112 of the present invention. The optical encoding pattern 112 includes a plurality of horizontal line segments 120 and a plurality of vertical line segments 122 that form a lattice pattern. The optical encoding pattern 112 also includes a plurality of optical markers 124 that are used to identify a predetermined position of the optical encoding pattern 112, such as the intersection position of the horizontal line segment 120 and the vertical line segment 122.

As illustrated in the figure, the optical markers 124 of a typical optical encoding pattern 112 are numbers from 1 to 225; however, other types of optical markers 124 can be employed. For example, the optical marker 124 may include a plurality of unique barcodes, a plurality of circles with different radii, and / or one of them. Other codes and symbols that uniquely identify a plurality of positions on the optical coding pattern 112 may be adopted.

FIG. 1D illustrates a portion 100 ′ of system 100, such as measurement unit 104, object holder 110, optical encoding pattern 112, camera unit 114, and measurement field of view 118, as illustrated in FIG. 1A. As illustrated in FIG. 1D, the measurement unit 104 obtains measurement data of the object 102 in the measurement field 118 (in the field of view range) of the measurement unit 104. The camera unit 114 acquires image data corresponding to the optical encoding pattern 112 formed on the lower surface of the object holder 110. The camera unit 114 may be any camera capable of acquiring a two-dimensional image of the optical encoding pattern 112. The camera unit 114 acquires image data corresponding to the visual field range 126.

For the purpose of explanation, in the object holder 110 of this example, it is assumed that the center of the measurement visual field 118 and the center of the visual field range 126 of the camera unit 114 coincide with the center of the object holder 110. The center of the object holder 110 is indicated by reference number 110A. In this example, when the object holder is positioned as shown in FIG. 1D, the pixel 123 corresponding to the center of the field of view range 126 of the camera unit 114 is the optical marker of the optical encoding pattern 112 as shown in FIG. 1E. Matches the center of “113”. As shown in FIG. 1C, “113” of the optical marker 124 corresponds to the optical marker 124 in the middle of the optical encoding pattern 112.

In the example of FIG. 1D, the center of the measurement visual field 118 coincides with the central pixel 123 of the visual field range 126 of the camera unit 114. In other embodiments, however, the center of the measurement field of view 118 may be offset from the center of the field of view range 126 of the camera unit 114 by a certain amount. This is a case where coordinates representing a part of the optical encoding pattern 112 at the center of the visual field range 126 of the camera unit 114 are converted into the central coordinates of the measurement visual field 118 of the reference coordinate system of the corresponding object holder 110 (described later). To be considered).

FIG. 1D shows arrows and axes labeled 125A, 125B, and 125C that display the reference directions of the measurement unit 104, the optical encoding pattern 112, and the camera unit 114 with respect to the respective coordinate systems.

3. process

FIG. 2 illustrates a method for obtaining a three-dimensional image of an object (eg, one or more teeth) and for obtaining an image of spatial characteristics (eg, spatial coordinates and orientation) associated with the object. 2 illustrates an exemplary process in one embodiment of the present invention. The process is performed by an optical position recognition system as shown by way of example in FIGS. 1A and 9. Reference is made to FIG. 2, which integrates FIG. 1A to FIG. 1E. The process starts at step S200. First, the object 102 is placed on or fixed to the upper surface of the object holder 110.

In step S202, the object holder 110 is placed at the selected position on the slide. In step S204, the measurement unit 104 acquires measurement data in the measurement visual field 118 where a part of the object is visible. (In other words, the measurement unit 104 acquires an image frame of a certain part).

In step S206, the camera unit 114 acquires image data including at least a part of the optical encoding pattern 112 in the visual field range (field of view) 126 of the camera unit 114 (in other words, the camera unit 114 acquires the optical encoding pattern 112). Digitally scan a part of it or get its image frame). In other words, the camera unit 114 acquires image data corresponding to a part of the optical encoding pattern 112 arranged to face the transmission unit 116 of the slide 106. In the preferred embodiment, the steps of S204 and S206 are performed simultaneously at each position of the object holder. However, other embodiments need not be.

In step S208, the processor of the system 100 uses the software module to process the image data acquired in step S206 to determine the value of one or more captured optical markers 124. The numerical value of the optical marker 124 identifies the intersection position of a line segment surrounding or adjacent to at least one captured optical marker 124 in the manner described below (eg, step S212). Used for. For example, if the optical marker 124 is a number, conventional optical character recognition (OCR) software may be used to measure the value of the captured optical marker 124. If the optical marker 124 is a barcode, conventional barcode reading software may be used to measure the value of the captured optical marker 124. If the optical markers 124 are circles with different radii, a software module may be used to determine the value of the captured optical marker 124.

In step S210, the processor of the system 100 (for example, the arithmetic unit 906 in FIG. 9 described below) uses the software module to process the image data obtained in the step of S206, and optical codes for the reference direction (for example, B125). A value representing the spatial characteristic (ie, orientation) of the digitized pattern 112 is determined. In one embodiment, a software module is used to determine the orientation of one or more optical markers 124, thereby determining the orientation of the optical encoding pattern 112.

As an example, if the optical markers 124 are circles each having a different radius, the software module may use the values of the at least two acquired optical markers 124 to determine the orientation of the optical encoding pattern 112. This is because in some cases, the orientation can be determined more accurately by using a plurality of symmetrical markers such as a circle. In an exemplary embodiment, the software module may make the determination based on the orientation of one or more line segments that intersect at the center of the two identified circular optical markers 124.

In an exemplary embodiment, a software module is used in step S210 to determine the orientation of one or more optical markers 124 and / or one or more horizontal segments 120 and / or perpendiculars 122. One direction is determined, thereby determining the direction of the optical coding pattern 112 relative to the reference direction. A technique for measuring the orientation of one or more line segments, eg, line segments 120 and / or 122, one or more optical markers 124, and the optical encoding pattern 112 is a method for determining the orientation of an object. Any of the techniques used appropriately for the above may be used.

The position information of the coordinates representing the predetermined positions of the intersections of the horizontal line segment 120 and the vertical line segment 122 of the optical encoding pattern is stored in a memory unit of the system 100 (for example, the secondary storage device 910 in FIG. ). The position information may be derived from calibration measurements or may be estimated from designations for the optical coding pattern 112. In one exemplary embodiment, the position information is stored in a memory unit prior to measurement data acquisition (eg, step S204) and used to correlate measurement data acquired during multiple measurements.

In step S <b> 212, the processor of the system 100 uses a software module to determine a spatial feature, such as coordinates corresponding to the center of a part of the optical encoding pattern 112, acquired from the visual field range 126 of the camera unit 114. In one exemplary embodiment, the software module may: (a) a specific intersection of the optical encoding pattern 112 observed within the specification of the field of view 126 of the camera unit 114; (b) an observed intersection. (C) the value of the optical marker 124 determined in step S208, and (d) the orientation of the optical encoding pattern 112 determined in step S210. Perform this measurement.

As an example, in one case, by identifying one intersection point adjacent to or surrounded by an optical marker 124 and deriving coordinates corresponding to the identified intersection point (position information of the intersection point), and Based on the coordinates and orientation of the optical encoding pattern 112 acquired in step S210, linear interpolation is performed by a known method, and the center of a part of the optical encoding pattern 112 acquired in the visual field range 126 of the camera unit 114 is obtained. Step S212 can be executed by determining the coordinates corresponding to.

In step S214, the processor of the system 100 uses the software module to obtain the optical encoding acquired in the field of view range 126 of the camera unit 114 corresponding to the center coordinate of the measurement field of view 118 in the coordinate system on the upper surface of the object holder 110. Coordinates associated with the center of a part of the pattern 112 are converted.

As an example of converting the coordinates of one coordinate system to another system, FIG. 6A shows the coordinates (20, −20) in the coordinate system 130 of the optical coding pattern 112, and FIG. 7A shows the top surface of the object holder 110. The coordinates (20, 20) converted into the reference coordinate system 140 are shown. The correspondence between the coordinates of the optical coding pattern 112 in the coordinate system 130 and the reference coordinate system 140 on the upper surface of the object holder 110 is based on the relationship between two predefined systems. Furthermore, the correspondence between the coordinates associated with the center of a part of the optical encoding pattern 112 acquired in the visual field range 126 of the camera unit 114 and the coordinates of the center of the measurement field 118 of the measurement unit 104 is 114 depends on the physical placement of the measurement unit 104 with respect to 114, and therefore step S214 may take that relationship into account.

In step S216, the processor of the system 100 uses the software module in a manner as described below, and uses each of the information obtained in step S210 and the measurement data obtained in step S204, for example. Perform spatial property transformations, such as associated coordinates. These coordinates may be determined prior to step 216, such as step S204, or may be measured at the beginning of S216. These coordinates are coordinates in the coordinate system of the measurement unit 104.

Referring to step S216, in that step, the software module determines the orientation of the optical encoding pattern 112 in the coordinate system of the optical encoding pattern 112 determined in step S210, and the reference coordinates of the upper surface of the corresponding object holder 110. Is converted into the orientation of the object holder 110 at. In other words, based on a predetermined mathematical algorithm that defines the relationship between the two coordinate systems, the orientation of the coordinate system of the optical coding pattern 112 is the corresponding orientation in the reference coordinate system on the top surface of the object holder 110. To be mapped. The software module uses the mathematical transformation to convert the coordinates associated with each piece of measurement data acquired from the coordinate system of the measurement unit 104 in step S204 to the reference coordinate system on the surface of the object holder 110. Convert to With this technique, when the measurement data is taken, the acquired data is taken in the reference direction wherever the object holder 110 is located.

According to this description, the transformations performed in steps S214 and S216 can be performed using any suitable translation algorithm operable in accordance with the present description, which can be readily understood by those skilled in the art.

In step S218, it is determined whether further measurements are to be performed (for example, whether it is necessary to obtain another desired part of the object 102). If further measurements are to be taken (“Yes” in step S218), step S202 is repeated and the object holder 110 moves to another selected position on the slide. And the process from S204 to S218 is repeated for this selected position. If no further measurement is required (“No” in step S218), the process ends at S220. All the frame data have coordinates converted in association with the reference coordinate system.

As a result, by using the acquired spatial characteristics such as coordinates, an aggregate of frames as measurement data can be formed, and synthesized measurement data for the object 102 is obtained. Therefore, step S220 may include a step of combining the measurement data obtained in step S204 based on the transformed coordinates obtained in step S216, whereby the object 102 is measured. A composite 3D image of the part is obtained. The composite image is generated by any appropriate frame matching technique, for example, an iterative nearest point (ICP) algorithm. However, in principle, the information from the camera unit 114 is obtained from the object 102. If it is sufficient to create a combined 3D display of the measurement sites, no frame association is necessary.

4). First example of typical measurement operation

3A-3J illustrate a first example of how the system 100 of FIG. 1A associates acquired data for an object 102 in an exemplary embodiment of the present invention (eg, FIG. 2 (see steps S206 to S216). First, the object 102 is fixed to the object holder 110 with an adhesive, for example. As shown for exemplary purposes in FIG. 3A, the object 102 has portions recognized as portions A-L.

A typical object holder 110 has a square cross-sectional area of 10 centimeters (cm) on each side. As shown in FIG. 3B, the reference coordinate system (X, Y) on the upper surface of the object holder 110 has an origin corresponding to the center of this square. The coordinates of this reference coordinate system (X, Y) are 1 cm apart. Therefore, as shown in FIG. 3B, each quadrant of the reference coordinate system (X, Y) is a square of 5 cm × 5 cm, and the range from 0 cm to 5 cm is the coordinate range. FIG. 3C shows the coordinates associated with the images of the parts A to L of the object 102 and the reference coordinate system (X, Y).

First example: Upper part

  As shown in FIG. 3D, the object holder is set so that the measurement unit 104 (not shown in FIG. 3D) can acquire measurement data in the measurement field 118 including the upper part of the object 102 (see, for example, step S204 in FIG. 2). 110 is positioned. FIG. 3E shows a coordinate system (X ′, Y ′) of the measurement unit 104 (not shown in FIG. 3E). The origin of the coordinate system (X ′, Y ′) of the measurement unit 104 corresponds to the center of the measurement visual field 118A. FIG. 3F shows images of measurement data of corresponding portions A to I of the object 102 and coordinates on the coordinate system (X ′, Y ′) of the measurement unit 104.

  FIG. 3G shows a portion 112A of the optical encoding pattern 112 and the corresponding field area 126A of the camera unit 114 (not shown in FIG. 3G) when the object holder 110 is positioned as in FIG. 3D. . As shown in FIG. 3G, when positioned as shown in FIG. 3D, the camera visual field region 126A is displayed in the coordinate system (X ”, Y”) of the optical coding pattern 112 with “83” and “83” of the optical marker 124. Located in the center of 98 ". This relationship can be better understood with reference to FIGS. 1A, 1C, and 1D. The optical encoding pattern 112 shown in FIG. 1C is positioned below the object holder 110 so as to face the camera unit 114 downward. In one example, the optical encoding pattern 112 is such that the top row (including the optical markers 124 shown in FIG. 1C including “1” to “15”) is located away from the support material 108, The orientation is adjusted so that the lower row (of the optical markers 124 shown in FIG. 1C, including “211” to “225”) is closest to the support material 108.

  As shown in FIG. 3G, the camera unit 114 (not shown) has six intersection points 127 and six optical markers 124 in the field of view 126A (see, for example, step S206 in FIG. 2). . The processor uses a software module to determine at least one value of the at least one optical marker 124 (see, eg, step S208 in FIG. 2). The processor also uses a software module to determine the orientation of the at least one optical marker 124, which is used to determine the orientation of the optical coding pattern 112 (eg, step S210 of FIG. 2). See).

  In addition, the processor uses the software module to obtain at least the coordinates of the intersection 127 of the line segment from the memory unit (for example, the second memory 910 in FIG. 9 described later), In the coordinate system (X ″, Y ″) of the optical coding pattern 112, it is used to determine the position coordinates of the portion 112A of the optical coding pattern 112 corresponding to the center of the field of view 126A (for example, FIG. (See step S212).

The processor uses the software module to change the orientation of the optical encoding pattern 112 determined in step S210 in the reference coordinate system (X, Y) on the top surface of the object holder 110 to the corresponding orientation of the object holder 110. And the center coordinates of the visual field 126A (determined by step S212) are converted into the coordinates of the center of the corresponding measurement visual field 118A (see, for example, step S214 in FIG. 2). In addition, the processor uses the software module to correspond the measurement data coordinates of the coordinate system (X ′, Y ′) of the measurement unit 104 to the reference coordinate system (X, Y) of the upper surface of the object holder 110. The transformation used to convert the coordinates to be generated is generated (see, for example, step S216 in FIG. 2). FIG. 3H shows measurement data images corresponding to portions A to I of the object 102 and their coordinates. These are corrected to the reference coordinate system (X, Y) on the upper surface of the object holder 110.

First example: Bottom

  Next, as shown in FIG. 3I, the measurement unit 104 (not shown in FIG. 3I) acquires measurement data within the measurement visual field 118B including the lower part of the object 102 (for example, the step of FIG. 2). (See S204), the object holder 110 is positioned. As shown in FIG. 3I, the object holder 110 is rotated by 90 ° from the orientation shown in FIG. 3D. FIG. 3J shows an image of measurement data corresponding to portions D to L of the object 102 and its coordinates in the coordinate system (X ′, Y ′).

  FIG. 3K shows the portion 112B of the optical encoding pattern 112 and the corresponding field of view 126B of the camera unit 114 (not shown in FIG. 3K) when the object holder 110 is positioned as in FIG. 3I. The camera unit 114 (not shown in FIG. 3K) can acquire four intersection points 127 and five optical markers 124 at which line segments intersect in the field of view 126B (see, for example, step S206 in FIG. 2). The processor uses a software module to determine at least one value of the at least one optical marker 124 (see, eg, step S208 in FIG. 2). The processor also uses a software module to determine the orientation of at least one optical marker 124 (eg, “113” of optical marker 124), which is used as the orientation of optical coding pattern 112 ( For example, see step S210 in FIG.

  In addition, the processor uses a software module from the memory portion (eg, the second memory 910 of FIG. 9 to be described later) to at least one intersection 127 of the line segment around “113” of the optical marker 124. , Which is used by the processor to determine the center coordinate of the field of view 126B in the coordinate system (X ″, Y ″) of the optical coding pattern 112 (eg, step S212 of FIG. 2). See).

  The processor uses the orientation of the optical encoding pattern 112 determined in step S210 and the center coordinates of the visual field 126A determined in step S212, and the reference coordinates of the orientation of the object holder 110 and the upper surface of the object holder 110. A software module is used that determines the coordinates of the center of the corresponding measurement field 118B in the system (X, Y) (see eg step S214 in FIG. 2).

  The processor uses the orientation of the optical encoding pattern 112 determined in step S210, the orientation of the object holder 110 determined in step S214, and the coordinates of the center of the measurement visual field 118B to obtain the measurement acquired by the measurement unit 104. Form a transformation used to convert the coordinates of each piece of data from the local coordinate system (X ′, Y ′) of the measurement unit to the reference coordinate system (X, Y) of the upper surface of the corresponding object holder 110. Software module (see, for example, step S216 in FIG. 2).

  FIG. 3L shows an image of measurement data corresponding to portions D to L of the object 102 and its coordinates. These are corrected from the coordinate system (X ′, Y ′) of the measurement unit 104 to the reference coordinate system (X, Y) on the upper surface of the object holder 110. As shown in FIG. 3L, the coordinates of portions D to L of the object 102 have been corrected due to the rotation of the optical encoding pattern 112 shown in FIG. 3I.

Thereby, measurement data is acquired for the upper part and the lower part of the object 102. The processor uses software modules to combine the upper and lower measurement data of the object 102 to create composite measurement data for the object 102. FIG. 3M shows an image of the combined measurement data corresponding to the parts A to L of the object 102 and the coordinates thereof. These coordinates are corrected to the reference coordinate system (X, Y) on the upper surface of the object holder 110.

Second example

4A-8E illustrate a second example of how the system 100 of FIG. 1A associates acquired data for an object 102 in an exemplary embodiment of the invention (eg, FIG. 2 (see steps S206 to S216). First, the object 102 is fixed to the object holder 110 with an adhesive, for example.

A typical object holder 110 has a square cross-sectional area of 10 centimeters on each side. The reference coordinate system 140 has an origin corresponding to the center of this square. The coordinates of this reference coordinate system are intervals of 1 millimeter (mm). 7A to 7D, each quadrant of the reference coordinate system is a square of 5 cm × 5 cm, and the coordinate range is from 0 cm to 50 mm (5 cm).

Second example: upper right part A

  As shown in FIG. 4A, the object holder 110 is positioned so that the measurement unit 104 can acquire measurement data in a visual field 118A including the upper right portion A of the object 102. FIG. 5A shows the portion 112A of the optical encoding pattern 112 and the corresponding field of view 126A of the camera unit 114 when the object holder 110 is positioned as shown in FIG. 4A.

  As described above with respect to step S208, the processor of the system 100 uses the software module to process the image data acquired by the camera unit 114 corresponding to the camera field of view 126A (FIG. 5A) and to obtain a camera field of view region 126A. At least one value of at least one of the optical markers 124 is determined. The processor uses one or more values of the optical marker 124 in the camera field of view 126A to identify intersections near the optical marker. As described above with respect to step S212, the processor determines the coordinates of this intersection, the position (indicated by reference numeral 132A in FIG. 6A), and the coordinates corresponding to the center of the field of view 126A of the acquired image.

In addition, the processor of system 100 uses software modules,
Then, the image data acquired by the camera unit 114 is processed to determine the orientation of at least one acquired optical marker 124 with respect to the reference direction indicated by the arrow 125 in FIG. 5A (for example, step S210 in FIG. 2 is performed). reference). In the illustrative example, optical marker 124 has no rotational offset relative to arrow 125, so the orientation of optical marker 124 shown in FIG. 5A with respect to the reference direction indicated by arrow 125 is zero.

  FIG. 6A shows an image of the coordinate system 130 of the optical encoding pattern 112. In this example, the coordinates of the location indicated by reference numeral 132A in FIG. 6A represent the center of the camera viewing area 126A and are determined to be (20, −20). The processor uses the software module to convert the coordinates of the location indicated by reference number 132A to a corresponding location in the reference coordinate system 140 on the top surface of the object holder 110. This is indicated by reference numeral 128A in FIG. 7A (eg, step S214). As shown in FIG. 7A, the coordinates of the location indicated by reference numeral 128A are determined as (20, 20).

The processor corrects the coordinates of each piece of measurement data 134A (FIG. 8A) acquired in the measurement visual field 118A of FIG. 4A to the corresponding coordinates of the reference coordinate system 140 on the upper surface of the object holder 110 (for example, step S216). For example, the coordinates of the center of the three-dimensional data 134A indicated by reference numeral 129A in FIG. 8A are converted so as to correspond to the coordinates of the center position of the measurement visual field 118A indicated by reference numeral 128A in FIG. 7A.

Second example: Lower right part B

  Next, as shown in FIG. 4B, the object holder 110 is positioned so that the measurement unit 104 can acquire measurement data in the visual field 118B including the lower right portion B of the object 102. FIG. 5B shows the portion 112B of the optical encoding pattern 112 and the corresponding field of view 126B of the camera unit 114 when the object holder is positioned as shown in FIG. 4B.

  As described above with respect to step S208, the processor of the system 100 uses the software module to process the image data acquired by the camera unit 114 corresponding to the camera field of view 126B (FIG. 5B) and to obtain a camera field of view region 126B. At least one value of at least one of the optical markers 124 is determined. As described above with respect to step S212, the processor uses one or more values of the optical marker 124 in the camera field of view 126B to locate the location corresponding to the center of the camera field of view region 126B (indicated by reference numeral 132B in FIG. 6B). ) And its coordinates.

In addition, the processor of system 100 uses software modules,
Then, the image data acquired by the camera unit 114 is processed to determine the orientation of the optical marker 124 with respect to the reference direction indicated by the arrow 125 in FIG. 5B (see, for example, step S210 in FIG. 2). In the illustrative example, the optical marker 124 of FIG. 5B is oriented with respect to the direction indicated by arrow 125, so the orientation of the optical marker 124 shown in FIG. 5B is zero with respect to the reference direction indicated by arrow 125. It is.

  FIG. 6B shows an image of the coordinate system 130 of the optical encoding pattern 112. In this example, the center of the camera viewing area 126B is indicated by reference numeral 132B in FIG. 6B and has coordinates determined as (20, 20). The processor converts the coordinates of the center of the camera viewing area 126B to a corresponding location in the reference coordinate system 140 on the top surface of the object holder 110. As shown in FIG. 7B, the corresponding location indicated by reference numeral 128B has coordinates determined as (20, −20).

The processor uses the software module to convert coordinates corresponding to the three-dimensional data 134B (FIG. 8B) acquired from the measurement field 118B of FIG. 4B into corresponding coordinates in the reference coordinate system 140 on the upper surface of the object holder 110. (For example, step S216). For example, the coordinates of the center of the three-dimensional data 134B indicated by reference numeral 129A in FIG. 8B are converted to correspond to the coordinates of the position of the center of the measurement visual field 118B indicated by reference numeral 128B in FIG. 7B.

Second example: Lower right part C

  Next, as illustrated in FIG. 4C, the object holder 110 is positioned so that the measurement unit 104 can acquire measurement data in a visual field 118 </ b> C including the lower left portion C of the object 102. FIG. 5C shows the portion 112C of the optical encoding pattern 112 and the field of view 126C of the camera unit 114 when the object holder is positioned as shown in FIG. 4C.

  As described above with respect to step S208, the processor of the system 100 uses software modules to process the image data acquired by the camera unit 114 corresponding to the camera field of view 126C and at least one in the camera field of view 126C. At least one value of one optical marker 124 is determined. As described above with respect to step S212, the processor uses one or more values of the optical marker 124 within the camera field of view 126C to use the coordinates (denoted by reference number 132C in FIG. 6C) that correspond to the center of the camera field of view 126C. ).

In addition, the processor of system 100 uses software modules,
Then, the image data acquired by the camera unit 114 is processed to determine the direction of the optical marker 124 with respect to the reference direction indicated by the arrow 125 in FIG. 5C. In a typical example, the optical marker 124 is oriented in the direction indicated by the arrow 125 in FIG. 5C, so the orientation of the optical marker 124 shown in FIG. 5C with respect to the direction indicated by the arrow 125 is indicated by the arrow 125. Zero for the reference direction shown.

  FIG. 6C shows an image of the coordinate system 130 of the optical encoding pattern 112. In this example, the center of the camera viewing area 126C is indicated by reference numeral 132C in FIG. 6C and has coordinates determined as (−20, 20). The processor converts the coordinates of the center of the camera field of view 126 </ b> C to a corresponding location in the reference coordinate system 140 on the top surface of the object holder 110. As shown in FIG. 7C, the corresponding location indicated by reference number 128C has coordinates determined as (−20, −20).

The processor uses the software module to convert coordinates corresponding to the three-dimensional data 134C (FIG. 8C) acquired from the measurement field of view 118C of FIG. 4C into corresponding coordinates in the reference coordinate system 140 on the upper surface of the object holder 110. (For example, step S216). For example, the coordinates of the center of the three-dimensional data 134C indicated by reference numeral 129C in FIG. 8C are converted to correspond to the coordinates of the position of the center of the measurement visual field 118C indicated by reference numeral 128C in FIG. 7C.

Second example: upper left part D

  Next, as illustrated in FIG. 4D, the object holder 110 is positioned so that the measurement unit 104 can acquire measurement data in the field of view 118 </ b> D including the upper left portion D of the object 102. For the purpose of this description, the object holder 110 has been rotated by 90 ° from the orientation of FIG. 4C. FIG. 5D shows the portion 112D of the optical encoding pattern 112 and the camera field of view 126D of the camera unit 114 when the object holder is in the position of FIG. 4D.

  As described above with respect to step S208, the processor of the system 100 uses software modules to process the image data acquired by the camera unit 114 corresponding to the camera field of view 126D and provide at least one in the camera field of view 126D. At least one value of one optical marker 124 is determined. As described above with respect to step S212, the processor uses one or more values of the optical marker 124 in the camera field of view 126D to use the coordinates corresponding to the center of the camera field of view region 126D (denoted by reference numeral 132D in FIG. 6D). ).

In addition, the processor of system 100 uses software modules,
Then, the image data acquired by the camera unit 114 is processed to determine the direction of the optical marker 124 with respect to the reference direction indicated by the arrow 125 in FIG. 5D. In this example, the object holder 110 is rotated clockwise by 90 ° with respect to the reference direction of the reference coordinate system 140, so that the reference direction of the coordinate system 130 of the optical encoding pattern 112 is shown in FIG. 5D. The direction of the optical marker 124 is 270 ° (90 ° counterclockwise) in the clockwise direction.

  FIG. 6D shows an image of the coordinate system 130 of the optical encoding pattern 112. In a typical example, the center of the camera viewing area 126D is indicated by reference numeral 132D in FIG. 6D and has coordinates determined as (−20, −20). The processor uses the software module to convert the coordinates of the center of the camera field of view 126D to a corresponding location in the reference coordinate system 140 on the top surface of the object holder 110. As shown in FIG. 7D, the corresponding location indicated by reference numeral 128D has coordinates determined as (−20, −20).

  The processor converts coordinates corresponding to the three-dimensional data 134D (FIG. 8D) acquired from the measurement visual field 118D of FIG. 4D into the reference coordinate system 140 (for example, step S216). For example, as shown in FIG. 8D, the coordinate indicated by reference number 129D corresponds to the center of the three-dimensional data 134D, and this is converted into the value of the coordinate indicated by reference number 128D in FIG. 7D.

  Further, based on the value of the orientation of the optical marker 124 with respect to the reference direction indicated by the arrow 125 in the coordinate system 130 of the optical encoding pattern 112, the coordinates of the three-dimensional data 134D are converted by 90 °. That is, the direction of the optical marker 124 of 270 ° with respect to the reference direction in the coordinate system 130 of the optical encoding pattern 112 is 90 ° with respect to the reference direction of the reference coordinate system 140 on the upper surface of the object holder 110. Correspond. In forming a composite three-dimensional image of the object 102, the three-dimensional data 134D is converted into the three-dimensional data 134A, 134B, and 134C by performing coordinate conversion of the three-dimensional data 134D based on the orientation of the marker 124. Make sure it is properly aligned.

The measurement data 134A, 134B, 134C, and 134D have three-dimensional data of portions A, B, C, and D of the object 102, respectively. As shown in FIG. 8E, the original coordinates of the measurement data 134A, 134B, 134C, and 134D were converted into corresponding coordinates in the reference coordinate system 140 on the upper surface of the object holder 110. Therefore, the coordinates of the measurement data 134A, 134B, 134C, and 134D shown in FIG. 8E are associated with the reference coordinate system 140 at this stage. The values of the measurement data 134A, 134B, 134C, and 134D and the corresponding coordinates in the reference coordinate system 140 are stored in a storage medium, and the storage medium includes composite three-dimensional data associated with the object. It is out.

5. Further examples

  Another embodiment of the present invention will be described.

  FIG. 11 shows an optical encoding pattern 1100 in an exemplary embodiment of the invention. The optical encoding pattern 1100 has a square shape having a height and a width of 90 mm, but other dimensions may be adopted. The optical encoding pattern 1100 can be installed in the system of FIG. 1A instead of the optical encoding pattern 112 described above. The camera unit 114 can visually recognize at least a part of the pattern 1100 through the transparent part 116 and acquire image data representing this part. As will be described below, this data is processed to determine the spatial characteristics of the optical coding pattern 1100. This characteristic is used as described above to determine the corresponding spatial characteristic of the object holder 110 relative to the measurement unit 104 in a manner similar to that described above.

  In one embodiment, with an accuracy of 5-10 microns (which may be 1 / 20-1 / 10 of a camera pixel), such as coordinates corresponding to a particular location of a portion of the optical encoding pattern 1100 Specific spatial characteristics can be determined. The direction of the portion of the optical coding pattern 1100 with respect to the reference direction can be determined with an accuracy of an angle of 0.04 °. Of course, in other embodiments, other shapes, dimensions, etc. may be used in place of those described above.

  An optical coding pattern is defined by a predetermined characteristic (same or different size, shape, color, radius, diameter, hollow shape, dense structure, distance between optical features, or characteristics thereof, depending on the optical features. The optical characteristic structure having other characteristics). In this illustrative example, optical coding pattern 1100 has, as an example, 1296 optical features (or optical markers) 1105 that are substantially circular and substantially dense. . However, in other embodiments, the optical encoding pattern 1100 may have a hollow structure or a substantially hollow circular structure. When the hollow circular structure is adopted, the characteristic value of the hollow circular structure can be determined based on the radius of the inner portion of the hollow circular structure and the radius of the outer portion of the hollow circular structure. Therefore, when using a circular cavity-like optical structure, the corresponding feature can be determined using two radius values, thereby comparing to a method using one radius value, Corresponding values can be determined with higher accuracy. In other embodiments, other types of shapes may be used in addition to the circular structure. Although the embodiment shown in FIG. 11 employs a substantially circular optical feature structure, the optical feature structure 1105 of various predetermined dimensions is used to locate a particular position on the optical coding pattern 1100. In order to be uniquely identifiable, the radius of each circular optical structure 1105 may have X (eg, 1) of a predetermined number Y (eg, 5) of discrete values. . Further, the optical encoding pattern in the above-described modification of the embodiment shown in FIG. 1C has, for example, a circular optical marker 124, each of which is used to identify a specific position of the optical encoding pattern 112. It may have its own radius that can. Furthermore, the optical encoding pattern 100 in the exemplary example of FIG. 11 has a horizontal line segment 120 and a vertical line segment 122 that form a lattice pattern, such as the optical encoding pattern 112 shown in FIG. 1C. Absent. However, in other embodiments, such line segments may be employed in the optical encoding pattern 1100.

  In other embodiments of the present invention, the distance between at least two optical structures (eg, the distance between one or more adjacent sets of optical structures (ie, the distance between optical structures)) is a predetermined number. You may have X pieces (for example, 1 piece) of the values of Y (for example, 5 pieces). In this embodiment, the distance between optical structures (features) may have an associated feature quantity, and this may be associated in a storage table that maps the distance between optical structures to the corresponding feature value. .

In yet another embodiment of the invention, the optical coding pattern may be provided with a plurality of optical feature structures disposed thereon. Each of the plurality of optical structures may have X (for example, one) of a predetermined number Y (for example, five) discrete colors. For example, in this example, the color value may have a corresponding feature value, and this may be associated in a storage table that associates the color value with the corresponding feature value. In other embodiments of the invention, the optical feature 1105 of FIG. 11 may have such a color.

  FIG. 11 also shows a camera field of view 1110 (ie, a region within the field of view of the camera unit such as the camera unit 114 of FIG. 1A, for example). The camera viewing area 1110 has a square shape with a height and a width of about 15 mm each, but other shapes and dimensions may be adopted. Preferably, the camera field of view such that each frame of the acquired image corresponding to the camera field of view 1110 has at least a complete N × M (eg, 3 × 3 array or other array) array of adjacent optical features 1105. The dimensions of region 1110 and optical feature 1105 are preselected. Such an array of adjacent optical feature structures 1105 can be used to determine the spatial characteristics of the optical encoding pattern 1100, as described below.

  An example of the optical encoding pattern 1100 has been described. A method for creating such an optical encoding pattern will be described below. FIG. 12 shows an example of a process for forming the optical encoding pattern 1100 shown in FIG. This process begins at step S1200. In step S1202, a substrate is supplied. The substrate may be formed of, for example, carbon steel, stainless steel, aluminum, brass, or copper. Of course, depending on the applicable design criteria, a substrate formed of other materials such as ceramic and paper can be used.

  In step S1204, coordinates are designated at a plurality of predetermined locations on the substrate in the first coordinate system. For example, this is performed so that each side corresponds to the lower left corner of the substrate of the optical coding pattern having a square shape of 90 mm square. In an example in which an array of 36 × 36 optical feature structures is evenly distributed over the optical coding pattern, the central portions of the optical feature structures are selected to be 2.5 mm apart. Therefore, in the first coordinate system, the coordinates of a plurality of predetermined positions on the substrate are, for example, (1.25, 1.25), (1.25, 3.75),. . . (88.75, 88.75). Each coordinate set corresponds to the center of the corresponding optical feature structure. In the first coordinate system, the coordinates of the positions of a plurality of predetermined positions on the substrate are stored in a memory unit (for example, the second memory 910 in FIG. 9).

  In step S1206, a master array of feature values is generated based on a plurality of predetermined positions (and coordinate sets) in step S1204 and an optical feature structure that can be observed in one camera field area (for example, camera field area 1110). The feature value is associated with a corresponding predetermined position and an optical feature structure arranged at the corresponding position (described later), and is supplied to the memory unit. For example, when a feature value is generated by a 36 × 36 master array, the feature value master array corresponds to 1,296 feature values, and each feature value is one of 1,296 coordinate positions. Associated with one. In one example, when each frame of image data acquired by a camera unit (for example, the camera unit 114 shown in FIG. 1D) includes at least an optical feature structure in a 3 × 3 subarray, these feature structures are Based on the information stored in the memory unit, the processor 906 (FIG. 9) can know that the feature value corresponds to a 3 × 3 sub-array of feature values (that is, a number corresponding to the optical feature structure). . Corresponding feature values form a 3 × 3 subarray. For example, a master array of feature values is initially populated with a 3 × 3 subarray, eg, [000; 000; 001] as the first subarray. A column with three elements is then added to the master array next to the first subarray. This results in an additional 3 × 3 subarray, which has a column with the added three elements and a second and third column of the first 3 × 3 subarray. This process is repeated until the master array is completely filled with feature values. Each 3 × 3 subarray in the master array is not identical to the other subarrays in the master array and preferably has no rotational symmetry.

  In step S1208, a plurality of optical structures are formed. Each optical feature structure corresponds to one of the feature quantities used to create the master array composed of feature values in step S1206. For example, FIG. 15A shows a table 1500 having five optical feature structures and corresponding feature values. Of course, however, the invention is not limited to such specific examples. Information as shown in the table 1500 may be stored in the memory unit. As can be seen from the example shown in the figure, the generated optical feature structure is circular. However, as described above, other shapes may be employed instead.

  In step S1210, each of the plurality of optical feature structures is arranged at each of a plurality of predetermined locations on the substrate based on the corresponding individual feature values of the master array of feature values. That is, based on the feature value included in the master array composed of the feature value generated in step S1206, the optical section is set so that the center of each optical structure corresponds to the coordinates of the predetermined corresponding location determined in step S1204. The feature structure is disposed on the substrate. For example, FIG. 15C shows a portion 1510 of an exemplary optical coding pattern that includes at least some of the optical feature structures shown in FIG. 15A. As noted above, FIG. 15A shows a feature structure associated with a corresponding feature value. The optical structures are arranged according to their corresponding feature values, which are included in the portion 1505 of the master array of feature values shown in FIG. 15B (ie, the feature value to which the center of each optical feature structure corresponds). To be placed at the coordinate position associated with the In step S1212, the process ends.

  In certain technology applications, this method works usefully, for example, ensuring the reproducibility of the optical feature structure at 1-2 microns. In at least some of the manufacturing methods used to create such optical coding patterns, it is not necessary to calibrate each created optical coding pattern independently. For example, in step S1210, an optical feature structure can be created on a substrate using computer numerical control (CNC) milling.

  Alternatively, a precision stamping or precision stamping process is used to “punch” a suitable material (eg, a substrate) to create an optical feature. Paperboard or flat plate can be used for the substrate. When a flat plate is used, the steel flat plate is useful because of its thermal expansion characteristics. In one example, the use of a flat plate having a thickness of about 500 microns results in a stable plate in the punching process. Precision punching is useful. This is because, when used, it is often necessary to calibrate only a precision punching tool and a small number of processing plates.

  In some cases, the diameter of the optical feature structure may be increased depending on the punching depth. That is, the diameter of the optical feature structure at the position of the bottom surface of the flat plate may be larger than the diameter of the corresponding optical feature structure at the top surface of the flat plate. If it is believed that the change in diameter may affect the accuracy of the image recognition process, the hole can be plugged in a later production process. For example, a hard plaster material may be used as a filler to fill the hole. The surface is then polished using a sharp blade edge. By selecting the appropriate color filler, the image contrast can be optimized. Similarly, an optical encoding pattern may be mounted on the lower surface of the object holder (for example, the object holder 110 shown in FIG. 1). Thus, the object holder itself and / or the plate may be colored so that the plate has a high contrast between the plate and the punched hole (which corresponds to the optical feature).

  Having described a method for creating an optical coding pattern in an exemplary embodiment, a method for determining the spatial characteristics of an optical coding pattern using a system such as that shown in FIG. 1A will be described. FIG. 13 illustrates an exemplary process for determining spatial characteristics corresponding to a portion of an optical coding pattern in accordance with an exemplary embodiment of the present invention. At least a part of the optical coding pattern is placed in the field of view of the camera unit. For example, the optical encoding pattern 1100 is the same method as shown in FIG. 1D described above (the same as the optical encoding pattern 1100 is used instead of the optical encoding pattern 112). It is installed on the lower surface of the object holder 110 facing the field of view of the camera unit 114.

  In step S1300, the process starts. In step S1302, image data corresponding to a part of the optical coding pattern (for example, 1100) is captured by the camera unit (for example, camera unit 114) in the same manner as described in relation to step S206 of FIG. A frame is obtained. For example, the camera unit 114 acquires image data of at least a part of the optical encoding pattern 1100 in the camera view area 1110 (shown in FIG. 11) of the camera unit 114 (that is, the camera unit 114 acquires the optical encoding pattern). Digitally scan a portion of 1100 or acquire its image frame). In other words, the camera unit 114 acquires image data corresponding to a part of the optical encoding pattern 1100 that is arranged to face the transparent portion 116 of the slide 106.

  In this example, the acquired frame of image data is a two-dimensional sub-array of acquired optical feature structures 1105. For example, the frame of the image data may correspond to an image acquired in the camera view area 1520 shown in FIG. 15E. In this example, it is shown that a total of 13 optical feature structures are acquired in the camera field of view 1520. As can be seen from FIG. 15E, nine feature structures are 3 × 3. Sub-arrays, that is, a central circle and eight feature structures adjacent thereto. That is, the optical feature structure acquired in the camera field of view 1520 is a 3 × 3 sub-array that includes nine optical feature structures in which each of the three optical structures is arranged in three rows. The corresponding optical feature structure forms each column of a 3 × 3 subarray.

  In step S1304, a feature value is determined for each optical feature structure of the optical feature structures forming the sub-array acquired in the camera field of view 1520 based on the respective characteristics of the feature structures. This determination may be made using at least one characteristic (eg, size, shape, color, radius, diameter, etc.) for each such optical feature structure using, for example, existing (or future developed) image processing techniques. Etc.). For example, if this property is a radius, by using existing or future digital image processing techniques that use edge detection, by examining the boundary pixels of the feature structure and fitting a circle to it , Can detect the radius. In this case, circular fitting is performed by using a solution of a linear set of equations. As described above, each radius determined in this way corresponds to the feature value based on the information stored in the memory unit. For example, a table (eg, FIG. 15A) stored in the memory unit can be accessed to obtain feature values corresponding to the radius determined for each optical feature structure (or directly corresponding to the feature structure itself). . FIG. 15F is an example of a 3 × 3 array 1535 formed from feature values, which corresponds to a 3 × 3 array of optical feature structures obtained within the camera field of view 1520 shown in FIG. 15E.

  In step S1306, in the first coordinate system, coordinates are determined for each central portion of the optical feature structure included in the sub-array of optical feature structures. For example, the first coordinate system is a coordinate system used by the camera unit 114 shown in FIG. 1A. FIG. 17A shows an example of the first coordinate system 1700. This first coordinate system 1700 is defined based on the camera view area 1520 shown in FIG. 15D. More specifically, the first coordinate system 1700 is defined such that the central portion of the central pixel 1525 of the camera view area 1520 shown in FIG. 15D is the coordinates (0, 0). In this example, the coordinates defined in step S1306 are determined within this coordinate system.

  The coordinates of the central portion of the optical feature structure included in the optical feature structure sub-array can be determined using, for example, image processing software that implements the Zhou operator. However, other procedures may be employed in other embodiments. The Zhou operator is discussed in a publication (Zhou, G., “Accurate determination of elliptic centers in digital imagery”, ASPRS Annual Convention, Vol. 4, March 1986, pp 256-264). The entire contents are incorporated herein by reference. FIG. 17B shows a table 1710 containing exemplary coordinates in the first coordinate system 1700. This is calculated in step S1306 in the coordinate system 1700 of FIG. 17A.

  In step S1308, a sub-array consisting of feature values is generated based on the feature values determined in step S1304 and the coordinates determined in step S1306. FIG. 15F shows an example of a 3 × 3 subarray 1535 of feature values, which corresponds to at least some optical feature structures included within the camera field of view 1520 shown in FIG. 15E. .

  In step S1310, the subarray of feature values generated in step S1308 is positioned in the corresponding master array of feature values (ie, the subarray is matched with the corresponding subarray in the master array). For example, as shown in the conceptual diagram of FIG. 16A, a 3 × 3 sub-array 1535 composed of feature values is positioned in a master array composed of feature values 1600.

  In step S1312, in the second coordinate system, the optical coding pattern (see step S1204) corresponding to the feature value generated in step S1308 and included in the feature value sub-array positioned in step S1310 is associated with a predetermined position. The coordinates are extracted from the memory unit (that is, the coordinates corresponding to the feature value matched in step S1310 are extracted). For example, the data structure corresponding to the master array formed from the feature values (see step S1206) is stored in a memory unit (for example, the second memory 910), which is associated with each feature value in the second coordinate system. The coordinate value at is included. The memory unit is accessed, and coordinates associated with one or more of the feature values included in the sub-array including the feature values generated in step S1308 are extracted. As an example, FIG. 16B shows an example table 1605 indicating coordinate values extracted from the memory unit in step S1312.

  In step S1314, one or more of one or more feature values included in the sub-array including at least one coordinate determined in step S1306 (in the first coordinate system) and the feature value generated in step S1308. Based on the coordinates (second coordinate system), at least one of some spatial characteristics (in the second coordinate system) of the optical coding pattern acquired in step S1302 is determined. This spatial property may be, for example, at least one of the coordinates and orientations determined as follows, with reference to FIG. 14, or other types of space associated with one or more acquisition portions of the optical coding pattern. Properties can be included.

FIG. 14 shows a more detailed example of the method when step 1314 is performed. The process of FIG. 14 begins at step 1400.

In step S1402, distortion is removed from the image acquired in step S1302. For example, the value of image distortion is determined prior to acquisition of the frame of image data in step S1302. Image distortion values can be obtained by inspecting an image of a known calibration object, a position value calculated from a feature structure of a calibration object, a known position value corresponding to a calculated position value of a feature structure ( Image distortion values or deviating values (ie, deviations from known and calculated position values) by comparison with position values determined using a high resolution measurement system Amount). The resulting deviation amount is stored in a lookup table and may be used to correct the acquired image. The image deviation may be used to remove distortion from the acquired image by existing (or later developed) image distortion removal techniques. For example, in April 2002, Journal of Electronic Imaging, Vol. 11, No. 2, pp. 157-176, published by Amidror, I, “Scattered Data Interpolation in Electronic Imaging Systems: Its Investigation (Scattered Data Interpolation”). Image distortion may be removed using a process such as that discussed in the paper "methods for electronic imaging systems: a survey"). This paper is incorporated by reference as a complete description of the entire project.

In step S1404, the coordinates (Xc, Yc) determined in step S1306 in the first coordinate system of the camera unit are converted into physical measurement units, for example, microns using, for example, the following equation (E1). In order to enlarge (or reduce) the correspondingly scaled coordinates (Xsc, Ysc), the camera unit is used at a magnification M. For example, the magnification M may be determined by examining a known calibration object or using other suitable techniques in the calibration process prior to acquiring the frame of image data in step S1302. For example, in a calibration step, an image of a known pattern may be acquired and compared to the actual size of the known pattern using a known calibration process. As described above, the scaled coordinates (Xsc, Ysc) may be derived by the following equation (E1).

In step S1406, the value of angle α is calculated. Here, the angle α represents an angle shift between the reference direction of the optical encoding pattern 1105 and the reference direction of the coordinate system of the camera unit 114. In one exemplary embodiment, step S1406 can be performed as follows. The center position of the optical feature structure of the 3 × 3 sub-array of optical feature structures is selected to form a rectangular lattice pattern. (Refer to step S1204) Information (for example, the coordinates of the center) defining these center coordinates is used to electronically match the rectangular lattice pattern to the center position. The angle of the grid pattern relative to the camera sensor is obtained as a fitting parameter for such a fitting. A modulo 90 ° arithmetic operator is applied to the obtained angle to determine the angle α. The ambiguity of the angle α may be removed using a lookup table in the master array of characteristic values. FIG. 15E shows a typical angle α formed between the reference orientation of the optical coding pattern indicated by arrow 1515 and the reference orientation of the camera unit coordinate system indicated by arrow 1530. In this example, the value of the angle α is determined to be 45 °. In other examples, the angle α can take other values.

In step 1408, the value of the angle α determined in step 1406 is used to rotate the scaled coordinates (Xsc, Ysc) to coincide with the reference coordinates of the first coordinate system of the camera unit. In the first coordinate system of the camera unit, the coordinates (Xscr, Yscr) rotated in this way may be determined by the following equation (E2).

In step S1410, coordinates (xscr, yscr) in the first coordinate system of the camera unit are converted into coordinates (x, y) in the second coordinate system of the optical coding pattern using the offset value. The offset value may be calculated from the distance of the center axis of rotation defined by the center of the 3 × 3 sub-array of the optical feature structure and the origin of the camera field of view (as shown in FIG. 17 as an example). The coordinates (0, 0) of the coordinate system 1700. This corresponds to the central pixel 1525 of the camera field of view 1520 described in FIG.

If the position of the optical characteristics in the optical coding pattern deviates from the normal position due to low accuracy in the optical coding pattern manufacturing process, these deviations may be corrected using the correction table. good. To do so, the optical coding pattern may be characterized by high resolution measurement before use. The value of the pattern deviation is preferably much smaller than the desired resolution of the entire system. For example, if the measurement unit has a resolution of the object surface data to the order of several microns, the pattern deviation should be even smaller.

In the method of the exemplary embodiment performed in step S1410, the coordinates (x in the second coordinate system corresponding to a specific position of a part of the optical coding pattern obtained in step 1302 are used. , Y) is determined in step S1410 using the following equation (E3). Here, the coordinates (xp, yp) may be acquired in step S1312, for example.

The process ends in step S1412, corresponding to step 1316 shown in FIG. In step S1412, information indicating one or both of the angle α and the coordinates (x, y) may be stored in the storage device or transmitted from the system of FIG. FIG. 18 shows an exemplary second coordinate system 1800 defined based on the optical encoding pattern 1100 shown in FIG. FIG. 18 shows a sign 1805 corresponding to the central pixel 1525 of the camera view range 1520 obtained in step S1302. In this example, in the second coordinate system, the coordinates (x, y) are measured as (68.75, 28.75).

The optical coding pattern 1100 described in FIG. 11 can be used to correlate measurement data in a manner similar to that described in FIG. Then, as described below, the information may be used together with the information obtained by the method of FIG. 13 in order to obtain a three-dimensional image of the object.

FIG. 19 illustrates an exemplary process according to an aspect of the present invention. The above process includes measurement data of an object (eg, one or more teeth) used to obtain a three-dimensional image of the object, and an object holder (eg, object holder 110 shown in FIG. 1A). ) To obtain spatial characteristics (eg, spatial coordinates and orientation). This process can be performed using an optical position recognition system, such as the system shown in FIGS. 1A and 9. In addition, at least some procedures described in FIGS. 12, 13, and 14 may be incorporated. Software for performing at least some of these procedures can be stored in a storage device, such as the second memory of FIG. It can also be executed by a processor such as processor 906 of FIG.

Referring to FIG. 19 in conjunction with FIGS. 1A, 1D and 11, the process begins at step S1900. First, the object 102 is placed on or fixed to the upper surface of the object holder 110. The optical encoding pattern 1100 is printed or fixed on the lower surface of the object holder 110 (instead of the optical encoding pattern 112 described in FIGS. 1A and 1D).

In step S1902, the object holder 110 is placed at the selected position of the slide. In step S1904, the measurement unit 104 acquires measurement data from the measurement range 118 where a part of the object 102 is visible (that is, the measurement unit 104 acquires an image frame of the part).

In step S <b> 1906, the camera unit 114 acquires image data of at least a part of the optical encoding pattern 1100 in the camera view range 1110 of the camera unit 114 and is associated with the acquired part of the optical encoding pattern 1100. In addition, a procedure for obtaining at least one spatial characteristic is executed. As a typical example of the present invention, step S1906 is executed by the executable steps from S1300 to S1316 of FIG. 13 described above.

In step S1908, the processor of system 100 (eg, processor 906 of FIG. 9) uses the software module to perform the transformation of coordinates associated with each piece of measurement data obtained in step S1904. These coordinates are coordinates in the coordinate system of the measurement unit 104, and may be measured prior to step S1908, for example, as in step S1904. The conversion in step S1908 can be performed using any suitable conversion algorithm. For example, as one exemplary embodiment of the present invention, it may be performed in a manner similar to step S216 as described above.

In step S1910, a determination is made whether further measurements are to be performed (eg, whether further measurements are required to obtain other suitable portions of the object 102). If further measurements are to be performed (“Yes” in step S1910), step S1902 is repeated, so that the object holder 110 is moved to another selected position on the slide. Steps S1904 to S1908 are repeated at the above position. If no further measurement is performed (“No” in step S1910), the process ends at S1912. At this point, all measurement data frames have transformed coordinates corresponding to the reference coordinate system of the optical coding pattern 1100.

Accordingly, a collection of frames of measurement data is formed using the acquired spatial characteristics such as coordinates, and composite measurement data of the object 102 is obtained. As such, for example, to obtain a three-dimensional composite image of the acquired portion of the object 102, step S1912 measures the measurement obtained in step S1904 based on at least one type of spatial characteristic determined in step S1906. A step of combining data can be included. The composite image display may be formed by an appropriate frame matching technique such as the Iterative Closest Point (ICP) algorithm. However, if the information from the camera unit 114 is sufficient to create a composite 3D display of the acquired portion of the object 102, in principle, no frame association step is required.

6). Typical system structure

The present invention (e.g., system 100, or components, or functions related thereto) may be performed by hardware, software, or a combination thereof, and by one or more computer systems or other processing systems. May be. Devices useful for performing some or all of the tasks of the present invention include general purpose digital computers or similar devices.

Indeed, in an exemplary embodiment, one or more computer systems have been used in the present invention to perform the functions as described herein. An example of such a computer system 900 is shown in FIG.

Computer system 900 includes at least one processor 904. The processor 904 is a communication infrastructure 906 (for example, a communication bus,
Crossover bar device or network). Various software embodiments have been described for this computer system 900, and with reference to this description, there is a method for implementing the invention using other computer systems and / or architectures. It will be easy to understand.

Computer system 900 includes a display interface (or other output interface) 902. The interface sends images, text, and other data from the communication infrastructure 906 (or frame buffer (not shown)) to the display on the display unit (or other output unit) 930.

Computer system 900 also includes main memory 908, which is preferably random access memory (RAM). Further, a second memory 910 may be included. The second memory 910 may include, for example, a hard disk drive 912 and / or a removable storage device drive 914 (for example, a floppy disk drive, a magnetic tape drive, and an optical disk drive). The removable storage device 914 can read and / or write to the removable storage medium 918 in a known manner. The removable storage unit 918 may be, for example, a floppy disk, a magnetic tape, an optical disk, or the like that can be read and written by a removable storage device 914 drive. The removable storage medium 918 can include a computer-usable storage medium that stores computer software and / or data.

The computer system 900 acquires an image and generates image data supplied to the processor 904, the main memory 908, and the second memory 910, or any one of them. 114). In addition, the computer system 900 includes a measurement unit 934 (for example, the measurement unit 104 in FIG. 1A) that acquires measurement data supplied to the processor 904, the main memory 908, and / or the second memory 910. Including.

In other embodiments, the second memory 910 may include other similar devices to allow computer programs or other instructions to be loaded into the computer system 900. Such devices include a removable storage medium 922 and an interface 920 (eg, a program cartridge and an interface cartridge similar to that used in video game systems); a removable memory chip (eg, an erase and programmable read only memory (“EPROM”)) or Any of programmable read-only memory (“PROM”) and associated memory sockets; and other removable storage media 922 and an interface 920 that allows software and data transfer from the removable storage media 922 to the computer system 900 It may be.

Computer system 900 also includes a communication interface 924 that enables software and data to be transferred between computer system 900 and external devices (not shown). Examples of the communication interface 924 include a modem, a network interface (for example, an Ethernet card), a communication port (for example, a universal serial bus (USB) port or a Firewire (registered trademark) port), and a PC memory card international association (“PCMCIA”) interface. Etc. Software and data transferred via the communication interface can be electrical, electromagnetic, optical, or other types of signals, and can be transmitted and / or received by the communication interface 924. It is. The signal is supplied to the communication interface 924 via a communication path 926 (for example, a communication path). The communication bus 926 is for signal transmission and may be a wire or cable, optical fiber, telephone line, cellular telephone link, wireless (“RF”) link, or the like.

The terms “computer program medium” and “computer usable medium” used in this description generally refer to, for example, a removable storage medium 918 used with a removable storage device drive 914, a hard disk installed in the hard disk drive 912, and Signal. These computer program products provide software to the computer system 900. The invention may be implemented and embodied as one or more such computer program products.

Computer programs (referred to as computer control logic) are stored in main memory 908 and / or second memory 910. The computer program may also be received via communication interface 924. Such a computer program is described therein at runtime, and the computer system 900 can perform the functions of the present invention as shown, for example, in FIGS. 2, 12, 13, 14 and 19. Like that. In individual items, the computer program, when executed, allows the processor 904 to perform the functions of the present invention. Therefore, such a computer program corresponds to a control device of the computer system 900.

In embodiments that implement the invention using software, the removable storage drive 914, hard drive 912, or communication interface 924 is used to store the software in a computer program product and load it into the computer system 900. Also good. Control logic (software), when executed by processor 904, causes processor 904 to perform the functions of the present invention as described herein.

In other exemplary embodiments, the invention is implemented primarily using hardware, such as hardware components such as application specific integrated circuits ("ASICs"). After reading this description, it will be readily apparent to those skilled in the relevant art how to implement a hardware arrangement or the like for performing the functions described herein.

In yet another exemplary embodiment, the present invention is implemented using a combination of both hardware and software.

As will be apparent to those skilled in the relevant art described herein, the present invention provides a single computer or a plurality of computers each programmed with control logic to perform the various functions described above. May be implemented by a computer system including:

The various embodiments of the present invention described above are given by way of example and not limitation. It will be apparent to those skilled in the relevant art that various changes in shape and detail (eg, different hardware, communication protocols, etc.) can be made without departing from the spirit or scope of the invention. Accordingly, the invention is not limited to any of the above-described exemplary embodiments, but is defined only in the following claims and their equivalents. By way of example, other embodiments may be ultrasound or other techniques in addition to optical imaging. It will also be appreciated that the steps and processes listed need not necessarily be performed in the order suggested. As an example, if the process is occupied by steps S208 and S210, these may be performed in the reverse order described above.

The above description describes an exemplary embodiment in which the camera unit acquires two-dimensional image data of the lower surface of the object holder and the spatial characteristics are determined based on the data. However, the present disclosure and the invention are not limited to the above functions. For example, to determine appropriate spatial characteristics based on acquired images from other sites, such as one or more aspects of the object holder and / or optical coding pattern. Are also included within the scope of the present invention. One of ordinary skill in the art will understand how to adjust the various steps of the approach described above to obtain optical properties based on acquired images in view of the present disclosure.

In addition, it should be understood that the attached diagrams that illustrate the functionality and advantages of the present invention are provided as illustrative examples. The structure of the present invention is sufficiently flexible and can be reconfigured and may therefore be used (operated) in ways other than those shown.

In addition, the purpose of the attached summary is disclosed here by the United States Patent and Trademark Office and the public, especially by scientists, engineers, and related technical experts who are not familiar with patents and / or legal terms and / or technical terms. It is to be able to make a quick judgment by simple research on the nature and nature of the subject technology. The summary is not intended to narrow the scope of the invention in any way.

Claims (11)

A method for evaluating an object placed on an upper surface of an object holder, the method comprising:
Obtaining at least one first image frame representing a captured portion of the object while positioning the object holder at each of a plurality of positions, the object being a first of the object holder; Arranged on the surface, the process,
Obtaining at least one second image frame representing at least a captured portion of the lower surface of the object holder while positioning the object holder at each of a plurality of positions;
Wherein based on the acquired second image frame, associated with the captured portion of the object, and determining the spatial characteristics including the spatial coordinates of its orientation of the object holder even without small, the ,
The determining process includes:
Determining values of a plurality of optical markers captured in each second image frame, wherein at least two of the optical markers have different characteristics including at least one of size and shape;
At least the one coordinate associated with the determined value of the optical markers, and determining at least Keru you to the lower surface, the orientation of the captured portion of the object holder,
Including a method.   The method of claim 1, further comprising creating composite data based on at least one of the acquired first image frame and the second image frame.   The method of claim 1, wherein the determining further comprises transforming coordinates associated with a captured portion of the object for each first image frame based on the orientation and the coordinates. . Determination of at least one of the coordinates and the orientation is:
In the first coordinate system, determining the coordinates of the central portion of each optical marker included in the plurality of optical markers,
Generating a plurality of feature values respectively associated with a central portion corresponding to one of the plurality of optical markers;
In the second coordinate system, a process of obtaining coordinates associated with one or more values among the plurality of feature values;
The method of claim 1 comprising:   The method of claim 4, wherein each of the plurality of optical markers has a circular shape, and each feature value is a corresponding radius of the plurality of optical markers.   The method of claim 1, wherein the plurality of optical markers form an array.   The method of claim 1, further comprising storing data indicative of the spatial characteristic.   The method of claim 1, further comprising transmitting a message including data indicative of the spatial characteristic.   The method of claim 4, wherein each feature value represents a distance between at least two optical markers. A system for evaluating an optical coding pattern, the system comprising:
An object holder in which the optical coding pattern is arranged in relation to its lower surface;
A measurement unit arranged to obtain at least one measurement frame of a captured part of the object arranged on the upper surface of the object holder;
An image acquisition unit arranged to acquire at least one frame of image data corresponding to a captured portion of the optical coding pattern including an array of optical feature structures, wherein the frame of image data is the optical feature An image acquisition unit comprising images of structures, wherein at least two of said arrays of optical feature structures have different properties including at least one of size and shape;
Associated with an array of feature values corresponding to the array of the optical characteristic structure, based on the feature value of the array of at least the optical characteristic structure associated with captured portion of the optical coding pattern, wherein the at no less a processor operative to determine the spatial characteristics including spatial coordinates of the object holder and its orientation, a processor, only including,
The processor further functions to generate composite measurement data based on at least one measurement frame acquired by the measurement unit and at least one frame of image data acquired by the image acquisition unit. ,system. A computer-readable recording medium storing control logic for causing a computer to evaluate an object placed on the upper surface of the object holder, the control logic comprising:
Computer readable program code for causing a computer to acquire at least one first image frame representing a captured portion of the object while positioning the object holder at each of a plurality of positions, A computer-readable program code disposed on a first surface of the object holder;
Computer readable program code for causing a computer to acquire at least one second image frame representing a captured portion of at least the lower surface of the object holder while positioning the object holder at each of a plurality of positions; ,
Based on at least one of the acquired first image frame and the second image frame, associated with the captured portion of the object, the spatial coordinates of the object holder even without least and its orientation Including computer readable program code for causing a computer to determine spatial characteristics including:
All of the codes are
Computer-readable program code for causing a computer to determine the values of a plurality of optical markers captured in each second image frame, wherein at least two of the optical markers have at least one of size and shape Computer readable program code having different characteristics including:
At least one coordinate associated with the determined optical marker value; and computer readable program code for causing a computer to determine the orientation of the captured portion on at least the lower surface of the object holder;
Including a computer-readable recording medium .

Images