JP5149288B2 - Composition of ANISOMETRIC texture (TEXTURE) (SYNTHESIS) - Google Patents

Abstract

An anisometric texture synthesis system and method for generating anisometric textures having a similar visual appearance as a given exemplar, but with varying orientation and scale. This variation is achieved by modifying the upsampling and correaction processes of the texture synthesis technique using a Jacobian field. The modified correaction process includes accessing only immediate neighbors of a pixel instead of non-local pixels. This constraint that only immediate neighbors be used also allows the generation of seamless anisometric surface textures. This is achieved by using indireaction maps containing indirection pointers that are used to jump from a set of pixels outside the boundary of a texture atlas chart to another chart. The system and method also includes an anisometric synthesis magnification technique that uses a Jacobian field to modify the magnification step of a synthesis magnification scheme and account for anisometry.

Description

  The present invention relates to texture synthesis.

  Texture synthesis involves automatically generating a large texture from a small sample image (called a texture sample or texture swatch). This sample-based texture synthesis creates content that is much larger than the content contained in the sample by taking the sample and generating additional content based on such sample. Using sample-based texture synthesis eliminates the need to explicitly store the entire content expanse. Instead, the content is generated “on the fly” from a much smaller sample.

  Traditionally, sample-based texture synthesis includes a modification process in which the neighborhood of each synthesized pixel is compared to the neighborhood of the sample. Next, the synthesized pixels are modified to recreate a synthesized neighborhood that is compatible with the sample content. For each pixel, a best-matching neighborhood is detected, and the current pixel is replaced with the best-matching pixel. The best matching neighborhood is determined by comparing the color of the pixels in the small pixel grid. For example, for each pixel, the 5 × 5 neighborhood of the pixel is examined, and the neighborhood error is the red, green, and blue (RGB: red, green, blue) color vector of the individual pixels contained in the 5 × 5 neighborhood. Is the sum of the errors.

  Texture synthesis may be isometric or non-isometric (meaning not “isometric”). In isometric texture synthesis, textures with the same pattern, scale, and orientation are generated. On the other hand, in non-isometric texture synthesis, the orientation of features (features) and local scaling can be performed in the generated texture.

  During the correction process, neighborhoods in the currently synthesized texture are collected and compared with the neighborhoods of the sample. In order to perform non-isometric synthesis, current techniques deform the collected neighborhood. The Jacobian inverse matrix (inverse Jacobian) is a transformation applied to neighboring coordinates before fetching colors. The Jacobian inverse matrix is a 2 × 2 matrix that transforms pixel neighborhoods using affine transformation. Specifically, when a Jacobian field is specified, pixel neighborhoods are transformed using a Jacobian inverse matrix. To perform the Jacobian inverse matrix correction, the orientation and scale of the pixel neighborhood is changed. That is, instead of obtaining a color at the original point in the neighborhood, a color is obtained with a new set of neighboring points transformed using the new orientation and scaling.

  Current non-isometric texture synthesis techniques have several drawbacks. The first drawback is the lack of memory coherency. Specifically, in some situations, the transformed neighborhood becomes larger and enlarged. This means that it is possible to access pixels that are far away rather than the immediate neighbor of the pixel. This can result in more random memory accesses, which can result in reduced efficiency and lack of coherency in memory accesses.

  A second drawback is that non-local pixels that are accessed during the correction process need to be resampled. Specifically, it is necessary to resample the pixels in the process of interpolating the pixel values after conversion. This work is necessary for the transformed pixels to be spread over a large area. However, this resampling process is inefficient and visual blurring may occur in the synthesized texture.

  A third drawback is the lack of texture coherency. Since the samples are widely spaced, the correction process does not depend on the value of adjacent pixels. This can significantly reduce texture coherency. Such incoherency reduces the functionality and effectiveness of the modification process, and in some cases the process may not converge.

  Another current non-isometric texture synthesis technique displays multiple versions of samples with different rotations and scales and selects from these sample versions based on the desired Jacobian at the current pixel. However, such an approach requires using a lot of memory to store and analyze many versions of the sample. Furthermore, it is difficult to realize texture coherency.

  In the following solution, some concepts explained in more detail in the best mode for carrying out the invention to be described later are introduced in a simplified form. The solutions presented below do not represent an important or essential function of the claimed subject matter. Nor is it used to limit the scope of the claimed subject matter.

  The non-isometric texture synthesis systems and methods disclosed herein include several techniques that enable the generation of high-quality non-isometric textures with high efficiency and minimal overhead. In general, non-isometric texture synthesis systems and methods synthesize new textures that are similar in appearance to a specified sample image or sample, but with different orientations and scales. These orientations and scales are specified by the Jacobian field. A Jacobian is a 2 × 2 matrix that represents a linear transformation of a particular point. Using this non-isometric texture synthesis system and method, it is possible to generate synthesized textures in a plane and on a surface.

  Non-isometric texture synthesis systems and methods include using a synthesis scheme that accesses only the immediate neighborhood of pixels. Using only direct neighbors improves both memory and texture coherency. Furthermore, there is no need to resample non-local pixels that are accessed during the modification process. It is assumed that the underlying compositing process depends on the sample pixel coordinates (not color). Therefore, the synthesized texture does not include color directly, but includes coordinates, and it is necessary to fetch the color of the pixel at this coordinate from the sample. Many texture synthesis schemes use this approach.

  In this setting, each of the coordinates in the sample is in the immediate vicinity of the designated pixel. Individual neighbors are used to predict candidates that will replace the current pixel. Instead of applying non-isometric transformations before accessing neighborhoods, non-isometric texture synthesis systems and methods can access direct neighborhoods and later apply transformations to sample coordinates obtained from direct neighborhoods. , Predict suitable candidates to replace the current pixel. In this way, only the immediate neighborhood is accessed, but the transformation is equally applied during the modification process.

  The goal is that if a pixel p is specified, a texture neighborhood is built around p and compared to the other neighborhoods in the sample using a best matching modification process. The direct neighborhood is obtained by using a direction component. When synthetic coordinates in the immediate vicinity are obtained, an offset is applied, and coordinates predicted by the present method are obtained as synthetic coordinates for the pixels themselves.

  The fact that only direct neighbors are used in this synthesis scheme also allows for the creation of a more seamless non-isometric surface texture. Specifically, if a parameterization is specified for a surface described as a triangle mesh and the texture domain of this surface, the parameterization partitions the mesh surface into charts, and each chart is an area of the texture domain Is mapped to The goal is to synthesize textures in the texture domain so that the resulting texture surface has a user-specified appearance. More precisely, the texture on the surface needs to match the Jacobian field specified by the user for the surface (in both rotation and scale). Part of the challenge is that the mesh parameterization breaks the surface into disjoint charts and that the surface texture is desirable to be seamless with respect to the boundaries of such charts. In order to construct such a seamless texture, it is necessary to include other chart samples in the vicinity of the synthesized pixels near the chart boundary. To accommodate these requirements, the non-isometric texture synthesis method provides the ability to jump from one atlas chart boundary to another by using an undirectional pointer.

  For a small set of pixels that are outside the boundaries of each chart, use a nondirectional pointer to point to the other pixels that are inside another chart. There is a small additional space around each chart (i.e., a gutter), and colors can be obtained according to these non-directional pointers. The function of constructing such a seamless surface texture directly utilizes the function of accessing the vicinity of the range where the non-isometric modification scheme is fixed.

  Non-isometric texture synthesis systems and methods further include non-isometric synthesis expansion techniques. Composite magnification includes obtaining a high resolution version of the same sample using the composite coordinates. Synthetic expansion can be used to hide sampling non-uniformities, such as parameterized distortion. The non-isometric texture synthesis method modifies the synthesis magnification scheme to include a Jacobian field. These Jacobians are used to change the expansion step of the synthetic expansion scheme and make it unequal.

  It should be noted that alternative embodiments are feasible, and that depending on the particular embodiment, the steps and elements described herein may be changed, added, or deleted. In such alternative embodiments, alternative steps and elements may be used and structural changes may be applied without departing from the scope of the present invention.

  Referring now to the drawings in which like reference numbers represent corresponding elements throughout.

  In the following description of the non-isometric texture synthesis method and system, reference is made to the accompanying drawings, which are elements of this specification. In the drawings, specific embodiments in which the present non-isometric texture synthesis method and system can be implemented are shown for purposes of illustration. It should be understood that other embodiments may be utilized and structural changes may be applied without departing from the scope of the claimed subject matter.

I. System Overview The present non-isometric texture synthesis system and method can be used to generate textures synthesized on a two-dimensional (2-D) plane or on a surface. The non-isometric texture synthesis system and method generally creates a new texture that is similar in appearance to the specified sample image or sample, but with a different orientation and scale specified by the Jacobian field. The non-isometric texture synthesis system and method described herein enables simple, efficient, and high quality 2-D non-isometric texture synthesis in both planes and surfaces. become.

  FIG. 1 is a block diagram illustrating an exemplary implementation of the non-isometric texture synthesis system disclosed herein. It should be noted that FIG. 1 only illustrates one of several approaches that can be implemented and used with this non-isometric texture synthesis system. This non-isometric texture synthesis system can also be implemented in other types of processing systems (eg, a central processing unit (CPU) or multi-core processing system).

  Referring to FIG. 1, the non-isometric texture synthesis system 100 is designed to operate on a computing device 110 (shown in dotted lines). The non-isometric texture synthesis system 100 may include various types of general purpose or special purpose computing system environments or configurations (including personal computers, server computers, handhelds, laptops, or mobile computers), or communication devices (eg, portable). Distributed computing environment including telephones and PDAs), multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and any of the systems or devices described above Note that it can work with The computing device 110 shown in FIG. 1 is only intended to represent any one of the foregoing and other types of computing system environments or configurations.

  As shown in FIG. 1, the computing device 110 includes a central processing unit (CPU) 120 and a CPU memory 130. The CPU 120 communicates with the video interface 140 via a high-speed bus (accelerated bus) 150. By way of example, the bus 150 may be, but is not limited to, an accelerated graphics port (AGP) or PCI Express specifically designed for 3-D graphics throughput requirements.

  The video interface 140 includes a graphics processing unit (GPU) 160 and a GPU memory 170. The GPU 160 is designed to speed up real-time 3-D graphics display. The GPU 160 can transfer data to the CPU 120 via the bus 150. The non-isometric texture synthesis system 100 is also in communication with a GPU 160 and a GPU memory 170. An isometric texture synthesis system 100 is shown in FIG. In the implementation shown in FIG. 1, system 100 includes unequal length texture synthesis system code 180 that resides on GPU 160 and unequal length texture synthesis data 190 that resides on GPU memory 170.

II. Outline of Operation FIG. 2 is a flowchart showing a general operation of the non-isometric texture synthesis system 100 shown in FIG. In general, in an unequal length texture synthesis method, an unequal length synthesized texture is generated from a sample image using a direct neighborhood of pixels and a Jacobian field. Non-isometric texture synthesis methods can be used to generate 2-D textures both on a plane and on a surface.

  More specifically, the present non-isometric texture synthesis method starts with the step of inputting an initialization image (box 200). Such an initial setting image includes composite coordinates. This is the result of the previous resolution level, the coarsest level due to the multi-resolution approach, and usually contains only zeros. The default image is the image at the previous resolution level, the coarsest level of the multi-resolution approach.

  A Jacobian field is also entered (box 205). Jacobian fields can be used to change the orientation and scale of the image. The Jacobian (usually represented by the symbol J) is a 2 × 2 matrix containing two vectors. These two vectors indicate how the input texture vector should be mapped to the image. In the Jacobian field, derivative information on how to define the original domain (in this case, a texture) on the image is recorded.

  The iterative process is started by boxing the pixel coordinates of the image according to the previous resolution level (box 210). In the first iteration, the upsampled image is the default image. In subsequent iterations, the upsampled image is the output of the previous iteration. Upsampling generates a current resolution level that is a finer resolution level than the previous resolution level. In other words, with a coarse-to-fine pyramid used in the multi-resolution approach, a series of iterations gradually increases the resolution level.

  Here, the pixel to be processed is selected and the immediate vicinity of the pixel to be processed is accessed (box 215). Next, the synthesized pixel coordinates stored in the immediate vicinity of the pixel are warped using the Jacobian field (box 220). Distortion means that the orientation and scale are changed as specified by the Jacobian field.

  Once the current candidate set of pixels is predicted using the distorted immediate neighborhood coordinates, a new neighborhood is constructed and a standard texture synthesis modification process is applied (box 225). A sample is entered into such a modification process (box 230). The process of such modification includes comparing the candidate neighborhood with the neighborhood computed for the current pixel. Note, however, that all of these neighbors are in an undistorted sample space.

  The output of each iteration is a composite image whose pixels contain the sample pixel coordinates at the current resolution level (box 240). Here, a determination is made whether the current resolution level is the finest (and thus the last) resolution level (box 245). If the current resolution level is not the finest resolution level, another iteration is performed. In this case, the current resolution level becomes the previous resolution level. That is, the pixel coordinates synthesized at the current resolution level become the pixel coordinates synthesized at the previous resolution level (box 250). Here, as described above, the iteration is executed until the finest resolution level is reached (box 255).

  If the current resolution level is the finest resolution level, the synthesized pixel coordinates are obtained (box 260). The pixel coordinates synthesized in this way can be processed by other applications or a corresponding non-isometric composite texture can be generated. To generate such an isometric composite texture, the synthesized pixel coordinates are first converted to a synthesized color by sampling a sample (box 265). Ultimately, such a conversion process results in a non-isometric composite texture being output (box 270).

III. Details of Operation Here, details of the non-isometric texture synthesis method will be described. FIG. 3 shows an example of the transition of the index map when the resolution is gradually increased in the special case of the non-isometric texture synthesis method shown in FIG. Note that the index map is an image of synthetic texture coordinates. In particular, FIG. 3 is a special case of non-isometric texture synthesis where the Jacobian is everywhere equal to a 2 × 2 identity 2 matrix. The index map is an image whose pixel values include coordinates that refer to the sample image domain. Such coordinates can be visualized as a color. For example, referring to FIG. 3, coordinates (0,0) are black 301, coordinates (1,0) are red 302, coordinates (0,1) are green 303, coordinates (1,1) are yellow 304 And colors are interpolated between them. An index map is a color map of pixel locations that uses color to indicate the location of the pixel color in the sample. In addition, FIG. 3 shows the resolution levels of an exemplary implementation with six resolution levels. As indicated by the long arrows, the resolution gradually becomes finer as the resolution level gradually increases.

  As shown in FIG. 3, the non-isometric texture synthesis method starts with the step of using an image (black image) 305 containing zero with a resolution level of zero. The iterative process shown in FIG. 2 is performed and the resulting first index map 310 is at resolution level 1. Note that each short arrow between index maps represents a single iterative process shown in FIG. For example, a short arrow 315 between the image 305 containing zero and the first index map 310 indicates that one iteration process of the unequal length texture synthesis method has been performed.

  By repeating this iterative process at progressively finer resolution levels, the resolution level is increased and a new index map is generated. Specifically, the iteration is executed at level 1 to generate the second index map 320 at level 2, the iteration is executed at level 2 to generate the third index map 325 at level 3, and the level 3 Is repeated to generate a level 4 fourth index map 330. A fifth index map 335 at level 5 and a sixth index map 340 at level 6 are generated in the same manner.

  FIG. 4 shows a texture image (that is, a synthesized color image) corresponding to the index map shown in FIG. The color of the texture image is obtained by accessing the sample color at the coordinates specified in the synthesized index map. The first texture image 400 corresponds to the first index map 310. Similarly, the second texture image 405 corresponds to the second index map 320, the third texture image 410 corresponds to the third index map 325, and the fourth texture image 415 corresponds to the fourth index map 330. , The fifth texture image 420 corresponds to the fifth index map 335, and the sixth texture image 425 (ie, the final composite texture) corresponds to the sixth index map 340.

  FIG. 5 is a block diagram illustrating an example of the non-isometric texture synthesis process shown in FIGS. 2, 3, and 4, where the current resolution level from the previous resolution level (level 1-1) of the multi-resolution approach. (Level 1). FIG. 5 shows the two main processes that are performed at each level (and at each iteration) of the multi-resolution approach. For example, referring to FIG. 3, FIG. 5 illustrates what processing is performed when transitioning from the first index map 310 to the second index map 320.

The first index map 500 is mathematically indicated by the term S _l-1 . However, S _l-1 is a composite coordinate output at the resolution level l-1 (that is, the previous resolution level). The sample 510 is mathematically represented by the term E [S _l-1 ]. However, E [S _l−1 ] shows the sample 510 as a function of the first index map 500 at the resolution level l−1. The combined output is an index, and the term E [S _l-1 ] indicates a color reference in the sample E when the coordinate term S is specified. Thus, an index map is generated by the synthesis process, which corresponds to the color E [S _1-1 ]. In other words, S is a composite index map, and E [S _l-1 ] is a color corresponding to such an index.

  As shown in FIG. 5, for each resolution level, the pixel coordinates represented by the first index map 500 are upsampled, and a second index map 520 representing the upsampled pixel coordinates is generated. By such up-sampling, the resolution level changes from level l-1 (previous resolution level) to level l (current resolution level). Next, the upsampled coordinates are modified to generate a third index map 530 representing the modified pixel coordinates at resolution level l.

A texture image corresponding to the index map is also shown in FIG. Specifically, the first texture map 540 corresponds to the second index map 520, and the second texture map 550 corresponds to the third index map 530. The index map is mathematically represented by the term S _l , and each texture image is represented as a function of the corresponding index map, ie the term E [S _l−1 ].

Non-isometric synthesis correction process Here, the non-isometric synthesis method will be described in more detail. As described above, the use of a Jacobian field allows local rotation and scaling of the texture according to Jacobian field J. The Jacobian field is used to change both the upsampling and modification processes of the non-isometric texture synthesis method.

Instead of defining sample textures with multiple versions that are variously distorted to be performed by other techniques, the present non-isometric texture synthesis method uses the synthesized neighborhood N _s before matching neighborhoods. Apply non-isometric distortion to. In other words, the synthesized neighborhood is rotated and scaled according to the Jacobian field.

  Further, as will be described below, texture and memory coherency is improved by using only the immediate neighborhood of the pixels. The underlying compositing process is based on pixel coordinates rather than sample colors. That is, the color is not directly included in the synthesized texture, but the coordinate is included, and it is necessary to fetch the color of the pixel of this coordinate from the sample. Each direct neighborhood of the designated pixel contains coordinates in the sample. Individual neighbors are used to predict candidates that will replace the current pixel. Instead of applying non-isometric transformations prior to accessing the neighborhood, non-isometric texture synthesis systems and methods are suitable for replacing the current pixel by accessing the neighborhood directly and later applying the transformation. Predict candidates. In this way, only the immediate neighborhood is accessed, but the transformation is equally applied during the modification process.

  The goal is that if a pixel p is specified, a texture neighborhood is built around p and compared to the other neighborhoods in the sample using a best matching modification process. The direct neighborhood is obtained by using the directional component. When synthetic coordinates in the immediate vicinity are obtained, an offset is applied, and coordinates predicted by the present method are obtained as synthetic coordinates for the pixels themselves.

  In non-isometric texture synthesis, such neighborhoods are constructed by simply looping in all directions with respect to the neighborhood, obtaining the resultant coordinates for these locations, and then only referring to the sample color. This constitutes the neighborhood of pixel p. Current non-isometric approaches change neighborhood access by considering the Jacobian inverse. This method has some of the disadvantages described above because access is extended by rotation and scaling.

  The non-isometric texture synthesis method disclosed herein accesses only the immediate neighborhood of pixel p. However, the direct neighborhood is obtained using the direction component. Therefore, the method moves to one of the eight immediate neighbors of p (but the closest in the selected direction). In this method, jumps to distant non-local locations (non-local with respect to pixel p) are not allowed. When the composite coordinates are obtained in the immediate vicinity, an offset is applied, and what is predicted by this method that the composite coordinates are of p itself is obtained. This offset is transformed by the Jacobian to account for unequal length distortion.

Mathematically, in the non-isometric texture synthesis method, a distorted synthesis neighborhood is defined as follows.
E [S [p + φ (hat)] − J (p) φ (hat) + Δ] (1)
However,
φ (hat) = φ / norm (φ) (2)
And φ = J ⁻¹ (p) Δ
It is.

  The direct neighborhood is specified by the neighborhood offset of the sample space. This is Δ in equation (1). Furthermore, φ is the offset of the neighborhood transformed by the Jacobian inverse matrix of the composite space. In the non-isometric texture synthesis method, instead of using non-local pixels, attention is paid to the direction of the vector, and the vector is normalized to the normalized distance 1. Here, the method determines which of the 8 direct neighbors within the 3 × 3 neighborhood is the direction to be examined by the method. The term S [p + φ (hat)] in equation (1) is this neighborhood, and the term φ (hat) is the directional component of the neighborhood position transformed by the Jacobian. This gives the location of any of the eight neighbors to access.

  The synthesized texture is then accessed in the immediate vicinity of the direction it is desired to sample. The next term in equation (1) is J (p) φ (hat). This term is the Jacobian at a particular pixel multiplied by the directional component of the nearby position transformed by the Jacobian. If this term is subtracted from the synthesized texture coordinates obtained from the immediate neighborhood of p, a prediction of the synthesized coordinates of p itself is obtained. Finally, by adding the neighborhood offset Δ of the sample space in the equation (1), the composite coordinates at the neighborhood offset Δ of the sample space of p are predicted.

  Directly adjacent composite coordinates S [p + φ (hat)] are accessed. The composite coordinates at the point p are predicted by subtracting J (p) φ (hat) from these coordinates. Further, when the composite coordinates at p are predicted using such direct neighbors, the corrected neighbor coordinates are predicted by adding the sample space offset, delta (Δ), to the composite coordinates.

  Delta (Δ) represents the neighborhood offset. In some embodiments, the neighborhood is a 5 × 5 neighborhood, while in other embodiments the neighborhood is a 3 × 3 neighborhood, and in some cases, four orthogonal points around the point p. For example, in the case of a 3 × 3 neighborhood, Δ = (− 1, −1), (−1,0), (−1,1), (0, −1), (0,0), ( 0, 1), (1, -1), (1, 0), (1, 1). As another example, a neighborhood composed of four orthogonal points is composed of Δ = (1, −1) (−1, 1) (−1, −1) (1, 1).

  In this non-isometric texture synthesis method, the coordinates of p + Δ are predicted. The neighborhood of 4 pixels is constituted by Δ = (1, −1) (−1, 1) (−1, −1) (1, 1). An important concept is that once a candidate for p is obtained in the sample space, a candidate at another location in the sample space is obtained by applying an arbitrary offset, delta (Δ). In this way, four candidates in the sample space, ie one candidate for every delta (Δ), are obtained.

  In addition, the value of a specific number of pixels per delta (Δ) is averaged, resulting in a better prediction of the appearance at p + Δ. This improves the quality of the neighborhood by averaging the contribution of a specific number of pixels per delta (Δ). In some embodiments, the number of pixels is equal to three. In other embodiments, the number of pixels may be equal to 3, more or less than 3.

Up-sampling process for non-isometric synthesis The up-sampling process of this non-isometric texture synthesis method is also modified using a Jacobian field. Thus, during the transition from coarse synthesis to fine synthesis, texture coherency is maintained by changing the individual upsampling paths as described unequal. Each child pixel inherits the parent's composite coordinates, and the Jacobian offset is a value obtained by multiplying the relative position of the Jacobian and the child. This concept is mathematically expressed by the following formula.
S _l [p]: = S _l−1 [p−Δ] + J (p) Δ, Δ = (± 1/2 ± 1/2) ^T (3)

Equation (3) shows that the composite S _{l of} p (ie, the child's composite coordinates) is equal to the parent's composite coordinates plus the Jacobian transformed relative position (Δ) of the child with respect to the parent. (Where S _l-1 represents a parent). Each coordinate of the 2D vector Δ is + ½ or −½, which means that there is a pyramid structure with two children per parent. Thus, the relative position of each child relative to its parent is +1/2 or -1/2 for each coordinate.

  FIG. 6 shows an example of a first result of the non-isometric texture synthesis disclosed herein. Specifically, the unequal length composite texture 600 is generated using the unequal length texture synthesis process and a sample 610. A non-uniform (non-constant) Jacobian field is specified throughout the non-isometric composite texture 600. This can be understood by paying attention to how the egg grows at the center of the non-isometric composite texture 600 and how it becomes smaller at the edge.

  FIG. 7 illustrates an example of a second result of the non-isometric texture synthesis process disclosed herein. The non-isometric composite texture 700 is generated using a non-isometric texture synthesis process and a sample 710. Also in this case, a non-constant Jacobian field is designated over the entire synthetic texture 700. This can be understood by paying attention to how the zebra stripes are straight and nearly vertical in the sample 710 and are deformed and rotated in the non-isometric composite texture 700.

Surface Texture Synthesis One important application of non-isometric texture synthesis is the generation of surface textures. Such applications will now be described in connection with various improvements and enhancements specific to such applications for non-isometric texture synthesis methods.

  The Jacobian field specifies how to deform the composite texture. In the surface case, it is desirable to deform the texture so that it looks correct when the texture is displayed on the surface. FIG. 8 shows the derivative of the Jacobian field used for the surface texture application. In general, a surface is described by parameterization. The surface M is parameterized in the parameter domain D, meaning that there are coordinates on the surface that indicate how 2-D should be planarized. This is a texture mapping process, where for every triangle on the surface, a portion of the image is taken from the texture domain. A parameterized Jacobian field can be determined for every point on the surface. This shows how the parameter domain is mapped to the surface. Since surface geometry can be complex, it is common for a surface to be parameterized into multiple charts as shown at 910. In other words, the surface is divided into a plurality of charts, each of which is individually mapped to a texture domain.

  As can be seen from FIG. 8, another input required is the texture orientation. This input can be specified by the user or performed automatically. If specified by the user, the user is required to draw an orientation on the surface using the user interface. As shown in FIG. 8, this ultimately results in specifying vectors t (tangent vector) and b (normal vector) at each point on the surface. You can perform this task using various types of user interfaces.

  The terms b (tilde) 800 and t (tilde) 810 shown in FIG. 8 are called “pre-images” by parameterizing the vectors t and b. Inputs are t and b (whether user specified or automatic), and b (tilde) 800 and t (tilde) 810 are pre-images that specify the mapping of these vectors t and b to the parameter domain. . The last Jacobian used for surface texture synthesis is the standard axis of sample E for b (tilde) 800 and t (tilde) 810.

820 and

Jacobian to map to 830. Thus, when a surface is specified, the two vectors t and b for each point are specified using the user interface or automatically. Next, the Jacobian field of the composite image is calculated using t and b. This is used as an input for the above-mentioned non-isometric texture synthesis method.

  Mathematically, the goal is a standard axis with sample surface vectors t and b

820 and

Using 830 to perform non-isometric texture synthesis within a locally identified parameter domain. Referring once again to FIG. 8, (t b) = J _f J ⁻¹ I is shown. Where J _f is the 3 × 2 Jacobian of the surface parameterization f: D → M, and J is the desired 2 × 2 Jacobian of the composite map S: D → E. Therefore, J is detected by the following equation.
^{J = (t b) + J} f = ((t b) T (t b)) -1 (t b) T J f (4)
However, “+” represents a pseudo inverse matrix. When (t b) is orthogonal, (t b) ⁺ = (t b) ^T. Since the parameterization is piecewise linear, J _f is piecewise constant within each triangle. On the other hand, the tangent space (tb) changes for each pixel.

  FIG. 9 shows an example of the result of performing surface texture synthesis using this non-isometric texture synthesis method. Specifically, the sample 900 is used as an input to the non-isometric texture synthesis, and an unequal-length synthetic texture 910 is generated. An isometric surface texture 920 is generated using the surface texture synthesis process described above and the non-directional map described below.

Undirected maps Current technology synthesizes textures on surfaces, usually by combining colors at the vertices of an irregular mesh. On the other hand, in this non-isometric texture synthesis method, the texture is directly synthesized in the parameter texture domain of the texture atlas parameterization. In order to construct a seamless texture on the discontinuous atlas of the parameterized chart of the surface texture, it is necessary to include other chart samples near the composite of pixels near the chart boundary. Thus, the non-isometric texture synthesis method provides a function of jumping from one atlas chart boundary to another texture atlas chart by using a non-directional pointer. In addition, the method includes an additional step of confirming that the chart is continuous. This is achieved using a non-directional map.

  For a small set of pixels that are outside the boundaries of each chart, use a non-directional pointer to point to other pixels that are inside another chart. There is a small additional space (ie gutter) around each chart, and colors can be obtained according to these non-directional pointers.

  Non-directional pointers use the ability to access the vicinity of a fixed extent with a fixed isometric modification scheme. By replacing each access S [p] with S [l [p]], samples are read from the entire chart using the directional map l for each level. These non-directional maps rely only on surface parameterization and are pre-computed by moving across the chart boundaries. At the time of constructing the atlas, a space for a non-directional pointer having a width of 2 pixels necessary for each chart is reserved.

  FIG. 10 illustrates a level 1-6 multi-resolution synthesis pyramid 1000 (first level 1010, second level 1020, third level 1030, fourth level 1040, fifth level 1050, and sixth level. 1060). As shown in FIG. 10, because the same atlas parameterization is used at all resolution levels, additional gutter space is reserved for non-directional pointers at the finest level of the pyramid. Specifically, as shown in FIG. 10, at levels 4 to 6 (1040, 1050, 1060), the gutter space around the image is gradually increased for non-directional pointers.

Non-isometric synthesis expansion One challenge in synthesizing textures within an atlas is that some parameterization distortion is often inevitable and undersampling regions occur. One way to hide such non-uniform sampling is to use a synthetic magnification scheme. Composite enlargement uses composite coordinates to access a high resolution version of the same sample.

The main concept behind at least one current composite magnification scheme is to use the composite coordinates S to access a higher resolution sample E _H. Specifically, the pixel value of the continuous coordinate p is obtained by the combination of the four closest synthesized pixels as follows.

However, w (Δ) = | Δ x | · | Δ y | is the weight of bilinear interpolation (bilinear interpolation). In surface texture synthesis, the goal is not necessarily to scale to a higher resolution sample E _H , but to restore details lost in the undersampling region.

  This unequal length texture synthesis method is made unequal by changing the synthesis expansion scheme described above. In other words, the Jacobian is introduced and used to change the expansion step in the synthetic expansion scheme described above. This is achieved by accessing the Jacobian map represented by

Non-isometric synthesis expansion can be performed with a surface shader when rendering is performed, which adds some cost.

  FIG. 11 shows an example of this non-isometric synthesis expansion. As shown in FIG. 11, the textured surface has a horse shape 1100. When using synthetic magnification including Jacobian information, a portion 1110 of the texture surface 1100 is magnified. In the central image 1120, a part 1110 is displayed without the composition enlargement. In the image 1130 on the right, an image obtained by applying composite enlargement to a part 1110 is displayed.

IV. Exemplary Operating Environment The present isometric texture synthesis system and method are designed to operate in a computing environment. The following description is intended to provide a simple and general description of a suitable computing environment in which the present isometric texture synthesis system and method may be implemented.

  FIG. 12 illustrates an example of a suitable computing system environment in which the present isometric texture synthesis system and method may be implemented. The computing system environment 1200 is only one example of a suitable computing environment and is not intended to indicate any limitation as to the scope of use or functionality of the invention. Neither should the computing environment 1200 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary operating environment.

  The non-isometric texture synthesis system and method can be applied to a variety of other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments and / or configurations suitable for use with the present non-isometric texture synthesis system and method include personal computers, server computers, handhelds, laptops or mobile computers, or Includes communication devices such as mobile phones and PDAs, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, any of the aforementioned systems or devices Examples include, but are not limited to, distributed computing environments.

  The system and method for non-isometric texture synthesis can be described in the general context of computer-executable instructions such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like, including those that perform specific tasks and those that implement specific abstract data types. The non-isometric texture synthesis system and method may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be stored in both local and remote computer storage media including memory storage devices. With reference to FIG. 12, an exemplary system for implementing the present isometric texture synthesis system and method includes a general-purpose computing device in the form of a computer 1210.

  The components of the computer 1210 may include a processing unit 1220 (eg, a central processing unit CPU), a system memory 1230, and a system bus 1221 that connects various system components including the system memory to the processing unit 1220. Yes, but not limited to them. The system bus 1221 may be any of various bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. These architectures include, for example, the ISA (Industry Standard Architecture) bus, the MCA (Micro Channel Architecture) bus, the EISA (Enhanced ISA) bus (VEC), and the VESA (Video Electronics Standard bus). A mezzanine bath), but is not limited thereto.

  Computer 1210 typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by computer 1210. Also included are both volatile and non-volatile media and both removable and non-removable media. By way of example, computer readable media may include, but is not limited to, computer storage media and communication media. Computer storage media is volatile and non-volatile, and removable and non-removable, incorporated in any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data Media.

  Computer storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, This includes, but is not limited to, a magnetic disk or other magnetic storage device, or any other medium that can store the necessary information and that is accessible from computer 1210. A communication medium embodies computer-readable instructions, data structures, program modules, or other data typically included in a modulated data signal, such as a carrier wave or other transport mechanism, and provides any information transmission. Contains media.

  It should be noted that the term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. Examples of communication media include, but are not limited to, wired media such as a wired network and direct wire connection, and wireless media such as sound, RF, and infrared. Any combination of the above should also be included within the scope of computer-readable media.

  The system memory 1230 includes computer storage media in the form of volatile and / or nonvolatile memory such as read only memory (ROM) 1231 and random access memory (RAM) 1232. A basic input / output system 1233 (BIOS: basic input / output system) including a basic routine for supporting information transfer between components of the computer 1210 at the time of startup or the like is normally stored in the ROM 1231. The RAM 1232 stores data and / or program modules that are normally accessible immediately from the processing unit 1220 and / or that are currently operating on the processing unit 1220. As an example, FIG. 12 shows an operating system 1234, an application program 1235, other program modules 1236, and program data 1237, but is not limited thereto.

  The computer 1210 may further include other removable / non-removable, volatile / nonvolatile computer storage media. FIG. 12 shows, by way of example only, a hard disk drive 1241 that reads or writes a non-removable non-volatile magnetic medium, a magnetic disk drive 1251 that reads or writes a removable non-volatile magnetic disk 1252, and a CD ROM. In addition, an optical disk drive 1255 for reading from or writing to a removable non-volatile optical disk 1256 such as an optical medium is shown.

  Other removable / nonremovable, volatile / nonvolatile computer storage media that can be used in an exemplary operating environment include magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tapes, solid state RAM Including, but not limited to, a solid state ROM. The hard disk drive 1241 is typically connected to the system bus 1221 via a non-removable memory interface such as interface 1240, and the magnetic disk drive 1251 and optical disk drive 1255 are typically connected via a removable memory interface such as interface 1250. To the system bus 1221.

  The above-described drive and the corresponding computer storage medium shown in FIG. 12 can store computer-readable instructions, data structures, program modules, and other computer 1210 data. For example, referring to FIG. 12, the hard disk drive 1241 stores an operating system 1244, an application program 1245, other program modules 1246, and program data 1247. Note that these components can either be the same as or different from operating system 1234, application programs 1235, other program modules 1236, and program data 1237. Here, different numbers are assigned to the operating system 1244, the application program 1245, the other program modules 1246, and the program data 1247 to indicate at least another copy. A user can enter commands and information into the computer 1210 using a keyboard 1262 and a pointing device 1261 (commonly referred to as a mouse, trackball or touch pad).

  Other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, wireless receiver, or television or broadcast video receiver. These and other input devices are often connected to the processing unit 1220 via a user input interface 1260 connected to the system bus 1221, for example, a parallel port, game port, or universal serial bus You may connect by other interfaces and bus structures, such as (USB: universal serial bus). A monitor 1291 or other type of display device is also connected to the system bus 1221 via an interface, such as a video interface 1290. Further, the computer may include an output peripheral device other than the monitor, such as a speaker 1297 and a printer 1296 that can be connected via the output peripheral interface 1295.

  Computer 1210 can operate in a network environment using logical connections to one or more remote computers, such as remote computer 1280. The remote computer 1280 may be a personal computer, server, router, network PC, peer device, or other common network node. Typically, many or all of the elements described above in connection with computer 1210 are included, but only memory storage device 1281 is shown in FIG. The logical connection shown in FIG. 12 includes a local area network (LAN) 1271 and a wide area network (WAN) 1273, but other networks may be included. Such networking environments are very common in offices, enterprise-scale computer networks, intranets, and the Internet.

  When used in a LAN networking environment, the computer 1210 connects to the LAN 1271 through a network interface or adapter 1270. When used in a WAN networking environment, the computer 1210 typically includes a modem 1272 or other means for establishing communications over the WAN 1273 such as the Internet. A modem 1272 (which may be internal or external) can be connected to the system bus 1221 using a user input interface 1260 or other suitable mechanism. In a network environment, program modules described in connection with computer 1210 or portions thereof may be stored in a remote memory storage device. As an example, FIG. 12 shows a remote application program 1285 resident in the memory device 1281, but is not limited thereto. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.

  The best mode for carrying out the invention described above is shown for the purpose of illustration and description. Many modifications and variations are possible in light of the above teaching. It is not intended to be exhaustive or to limit the subject matter described herein to the particular forms disclosed. Although the subject matter has been described in language specific to structural features and / or methodological operations, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. I want you to understand. Rather, the specific features and acts described above are disclosed as example implementations of the claims appended hereto.

FIG. 2 is a block diagram illustrating an example implementation of the non-isometric texture synthesis system disclosed herein. It is a flowchart which shows the general operation | movement of this non-isometric texture synthesis system shown in FIG. In the special case of this non-isometric texture synthesis method shown in FIG. 2, an example of the transition of the index map (index map is an image of synthesized texture coordinates) when the resolution is gradually increased is shown. FIG. It is a figure which shows the texture image (namely, the synthesized color image) corresponding to the index map shown in FIG. FIG. 5 is a block diagram illustrating an example of a special case of the non-isometric texture synthesis process shown in FIGS. 3 and 4. FIG. 6 is a diagram illustrating an example of a first result of an isometric texture synthesis process disclosed herein. FIG. 6 is a diagram illustrating an example of a second result of the non-isometric texture synthesis process disclosed herein. FIG. 6 is a diagram showing a Jacobian field derivative used in a surface texture application. It is a figure which shows the example of the result of having performed surface texture synthesis | combination using this non-isometric texture synthesis method. FIG. 6 is a diagram illustrating a plurality of levels of a multi-resolution synthesis pyramid. FIG. 4 illustrates an example of this non-isometric synthesis expansion scheme. FIG. 2 illustrates an example of a suitable computing system environment in which the present isometric texture synthesis system and method may be implemented.

Claims (20)

A method of synthesizing non-isometric textures,
Inputting a Jacobian field in pixels and a synthesized pixel coordinate image at a previous resolution level;
A step of selecting the direction to be examined;
Determining a neighborhood offset vector transformed with the Jacobian inverse matrix of the composite space using the neighborhood offset of the sample space and the Jacobian inverse matrix at the pixel;
Normalizing a neighborhood offset vector transformed with a Jacobian inverse of the composite space to a normalized distance of 1;
Determining which of the eight immediate neighbors of the pixel to access based on the normalized distance of a neighborhood offset vector transformed with a Jacobian inverse of the composite space;
Accessing one of the eight direct pixel neighborhoods closest in the selected direction to the pixel being processed from the synthesized pixel coordinate image at a previous resolution level; The synthesized pixel coordinate image includes pixel coordinates that do not directly include a pixel color, and wherein the pixel color is retrieved in a sample texture;
And forming the Jacobian using field to distort the direct pixel neighborhood of the pixel coordinates, the distorted pixel coordinates were used for the synthesis of the anisometric texture in the pixel,
Using a general-purpose computing device having a processor for performing. The method of claim 1, further comprising selecting at least one candidate pixel from the sample texture using the distorted pixel coordinates .   The method of claim 2, wherein selecting the candidate at least one pixel further comprises using a correction technique.   4. The method of claim 3, wherein the correction technique is a per-pixel neighborhood matching correction technique (225).   5. The method according to claim 1, wherein the non-isometric composite texture is present on a surface, and the step of fitting the non-isometric composite texture to the Jacobian field on the surface is further included. Including. 6. A method according to claim 5, wherein
Directly synthesizing the non-isometric composite texture in a parameter texture domain of a texture atlas parameterization having a plurality of texture atlas charts;
Using a non-directional pointer that provides the ability to jump between the plurality of texture atlas charts;
Further comprising a method. The method of claim 6, comprising:
Selecting a set of pixels near a chart boundary in one of the charts;
Using the non-directional pointer to point to another pixel within another one of the chart;
Further comprising a method.   8. A method as claimed in any preceding claim, wherein the non-isometric composite texture is augmented using the Jacobian field and composite magnification technique to augment the non-isometric composite texture. The method further comprising generating a texture.   9. The method of claim 8, further comprising performing the synthesis magnification technique in a surface shader during rendering. A computer-readable storage medium having stored and encoded computer-executable instructions, said computer-executable instructions for synthesizing a texture on a surface using a sample;
Specifying a Jacobian field in pixels;
Selecting a direction to examine based on the direction component;
Determining a neighborhood offset vector transformed with the Jacobian inverse matrix of the composite space using the neighborhood offset of the sample space and the Jacobian inverse matrix at the pixel;
Normalizing a neighborhood offset vector transformed with a Jacobian inverse of the composite space to a normalized distance of 1;
Determining which of the eight direct neighbors of the pixel to access based on the normalized distance of a neighborhood offset vector transformed with a Jacobian inverse of the composite space;
Accessing one of the eight direct pixel neighbors closest to the pixel being processed in the direction of the directional component, wherein the synthesized pixel coordinate image does not directly contain a pixel color. The pixel color includes pixel coordinates retrieved in a sample texture;
Distorting the upsampled pixel coordinates of the pixel and its immediate pixels using the Jacobian field;
Modifying the distorted upsampled pixel coordinates using the Jacobian field to allow rotation and scaling of the coordinates of the pixels and neighboring pixels;
Parameterizing the surface across the parameter domain by mapping a parameter domain to the surface;
A computer readable storage medium comprising:   11. The computer readable storage medium of claim 10, further comprising receiving input as to how the texture should be directed on the surface, the input comprising: (a) each point on the surface. A computer readable storage medium comprising at least one of: (b) a normal vector at each point on the surface.   12. The computer readable storage medium of claim 11, further comprising the step of locally identifying the tangent vector and the binormal vector with respect to a standard axis of a sample. A computer-readable storage medium of claim 10, using a non-directional pointer, further comprising forming a seamless texture on the surface of discrete atlas parameterized chart, computer readable Possible storage medium. A computer readable storage medium according to claim 13, comprising:
Forming a gutter of additional space around each of the parameterized charts of the discontinuous atlas ;
Using the non-directional pointer in the gutter to point to other pixels within another chart;
A computer-readable storage medium further comprising:   15. The computer readable storage medium of claim 14, further comprising using a synthetic magnification technique, the synthetic magnification technique being modified by the Jacobian field to obtain a higher resolution version of the sample. Computer readable storage medium. A computer-implemented method for synthesizing a non-isometric composite texture from a sample comprising:
Inputting a Jacobian field J (p) at pixel (p) and an initialization image at a previous resolution level;
Up-sampling pixel coordinates at the previous resolution level to generate a current resolution level having a higher resolution than the previous resolution level;
Define φ as the neighborhood offset vector transformed with the Jacobian inverse matrix of the composite space, where φ = J ⁻¹ (p) Δ, Δ is the sample space neighborhood offset, and J ⁻¹ (p) is the pixel A step that is an inverse Jacobian of (p);
Normalizing a neighborhood offset vector transformed with a Jacobian inverse of the composite space to a normalized distance of 1;
Determining which one of the eight direct neighbors of the pixel to form a pixel neighborhood to access based on the normalized distance of a neighborhood offset vector transformed with a Jacobian inverse matrix of the composite space When,
Accessing one of the eight direct pixel neighborhoods of the pixel that is closest in a selected direction to the pixel being processed;
Distorting the upsampled pixel coordinates near the pixel using the Jacobian field;
Obtaining the non-isometric composite texture using the sample and the distorted upsampled pixel coordinates;
Using a computer having a processor to perform.   17. The method of claim 16, wherein modifying the distorted upsampled pixel coordinates using the Jacobian field to allow rotation and scaling of the coordinates of the pixels and neighboring pixels. Further comprising a method. The method of claim 17, comprising:
A step of parameterizing the sub Fesu over parametric meter domain,
Determining a Jacobian field of the parameter domain for every point on the surface to map the parameter domain to the surface;
Further comprising a method.   19. The method of claim 18, further comprising forming a seamless non-isometric composite texture on the surface using an undirected map.   20. The method of claim 19, wherein the synthesis expansion technique is modified using the Jacobian field to account for unequality in the non-isometric composite texture to obtain a higher resolution version of the sample. The method further comprising the step of:

Images