JP6554980B2 - Image processing system, method, and program - Google Patents

Description

  The present invention relates to an image processing system, a method, and a program, and more specifically, an image processing system, a method, and an image processing system for displaying a texture of a material of a display object, which can display a combination of a plurality of elements collected from a plurality of computers. And programs.

  With recent developments in manufacturing technology, communication technology, and information processing technology, portable terminals such as smartphones and tablet terminals are used in various scenes. In addition to the touch panel display and communication interface, the adaptation scene of the composite portable terminal having various functions such as GPS, gyro sensor, camera, etc. is expected to expand further in the future.

  For example, in the past, products that are the subject of commercial transactions have been introduced as photographs and illustrations in pamphlets on paper media. For building materials that are difficult to perceive the texture of the material in photographs and illustrations (color samples), they were introduced in the form of a sample collection of small pieces of the real thing.

  On the other hand, there has been proposed a method and apparatus for displaying an image of the texture of a subject (material) such as the glossiness of the subject surface and the fine texture of the fiber weave (see, for example, Patent Document 1). According to the disclosure of Patent Document 1, a plurality of original images are obtained by changing the illumination position and / or illumination direction of a light source and generated based on a plurality of original images for one stationary subject having different specular reflection states. A plurality of frame images are created, and a plurality of frame images are continuously displayed. A gloss signal is generated by using the specular reflection image signal and the diffuse reflection image signal of the subject, and this gloss signal is generated. Created by windowing the signal.

  Furthermore, a technique for adding a sense of depth to a still image by applying a parallax effect or the like and changing the texture of the material has been put into practical use.

  Those who wish to purchase a house such as a condominium can go to the model room and experience the floor plan of the real thing and the texture of the building materials. It is known to reproduce a house specification including a floor plan by computer graphics (CG) (for example, refer to Patent Document 2). Through a virtual model room provided by a home supplier, a home purchase applicant can experience a simulated home specification including a floor plan.

JP 2003-132350 A (refer to summary, claims 1, 2, etc.) Japanese Patent Laying-Open No. 2004-5216 (see paragraph 0037, FIG. 3)

  However, there is a problem that even if it is a paper medium or an electronic medium, it is impossible to express a change in appearance (texture) of the material according to a change in observation environment or a change in the direction of the material. Furthermore, the appearance of the material when placed in an actual space cannot be easily imagined. In addition, no system has been proposed that can simulate the texture and appearance of materials such as walls and floors placed in the space.

  Furthermore, it is impossible to simulate the texture of the material by freely combining the color, shape, and reflection characteristics of the material. The same applies to not only those who wish to purchase a home but also home suppliers (interior designers).

  Therefore, it is possible to freely combine multiple elements (that is, the color, shape, and reflection characteristics of the material), and to change the appearance (texture) of the material in response to changes in the environment and the orientation of the material. It is desirable to provide a system that can express and further experience the texture and appearance of the material when placed in space.

  The present invention has been made in view of such problems, and the object of the present invention is to combine a plurality of elements collected from a plurality of computers, and to respond to changes in the observation environment and changes in the orientation of the material. An object of the present invention is to provide an image processing system, a method, and a program that display an image of the texture of a material and allow a simulated experience of the texture and appearance when the material is arranged in a space.

  In order to achieve such an object, a first aspect of the present invention is an image processing system connected to a plurality of computers. The image processing system is applied to the space, an element collecting means, an element combination means, a first display means for displaying an image of a space, a first rendering means for rendering display data of the space, and the space. A second display unit that displays an image of the material; and a second rendering unit that renders display data of the image of the material. The element collection unit receives a material image from each of the plurality of computers. Configured to collect at least one of color characteristics, shape characteristics and reflection characteristics for each pixel, the element combination means combining the color characteristics, shape characteristics and reflection characteristics collected by the element collection means; Material information relating to the texture of the material is determined, and the second rendering means includes illumination information, the element The display data of the material is rendered on the basis of the material information on the texture of the material determined by the user combination unit, the lighting, the display surface of the second display unit, and the relative positional relationship between the viewers. And the second display means is configured to display an image of the material using the rendered display data of the material, and the first rendering means has an instruction to adapt the material to the space. In response to the information, the display data of the space to which the material is applied is rendered based on the spatial illumination information of the space and material information on the texture of the material, and the first display means is configured to render The image of the space is displayed using the displayed display data of the space.

  According to a second aspect of the present invention, an element collection unit, an element combination unit, a first display unit that displays an image of a space, a first rendering unit that renders display data of the space, and the space It is an image processing method in an image processing system including second display means for displaying an image of a material to be applied and second rendering means for rendering display data of the image of the material. The image processing system is connected to a plurality of computers, and in this method, the element collection unit is configured to detect color characteristics, shape characteristics, and reflection characteristics of each pixel of a material image from each of the plurality of computers. Collecting at least one, the element combination means combining the color characteristics, shape characteristics and reflection characteristics collected by the element collection means to determine material information relating to the texture of the material; The rendering means is based on illumination information, material information on the texture of the material determined by the element combination means, and illumination, a display surface of the second display means, and a relative positional relationship between the viewers. Rendering the display data, and the second display means uses the rendered display data of the material Displaying the image of the material; and in response to an instruction for adapting the material to the space, the first rendering unit displays the material based on spatial illumination information and material information on the texture of the material. Rendering the applied display data of the space; and displaying the image of the space using the rendered display data of the space.

  A third aspect of the present invention is an image processing program. This program is a program that causes a computer to function as the image system of the first aspect.

  As described above, according to the present invention, it is possible to freely combine a plurality of elements collected from a plurality of computers, and image the texture of the material according to changes in the observation environment and the orientation of the material. It is possible to provide an image processing system, a method, and a program for display.

1 is a schematic diagram of an image processing system according to an embodiment of the present invention. It is a figure explaining the space which displays an image with the image processing system of one Embodiment of this invention. It is a figure for demonstrating the user interface of the image processing apparatus of the image processing system of one Embodiment of this invention. It is a functional block diagram of the server of the image processing system concerning one Embodiment of this invention. It is a functional block diagram of the image processing apparatus of the image processing system concerning one Embodiment of this invention. It is a functional block diagram of the virtual space image processing apparatus of the image processing system of one embodiment of the present invention. It is a figure for demonstrating one Embodiment of this invention. It is a figure for demonstrating one Embodiment of this invention. It is a figure for demonstrating one Embodiment of this invention. It is a figure for demonstrating one Embodiment of this invention. It is a flowchart of a process of the server of the image processing system which concerns on one Embodiment of this invention. It is a flowchart of a process of the image processing apparatus of the image processing system which concerns on one Embodiment of this invention. It is a flowchart of a process of the virtual space image processing apparatus of the image processing system which concerns on one Embodiment of this invention. It is a flowchart of the process which produces | generates the manufacturing data of the image processing system which concerns on one Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The image processing system, method, and program described below are suitable for an electronic sample book for building materials used in indoor and outdoor spaces such as houses. That is, according to the image processing system, method, and program of the present embodiment, it is possible to display an image of the texture of the building material according to the change in the observation environment and the change in the direction of the building material. Sample collections of small pieces or simple color samples can be an unnecessary alternative.

  The present invention is not limited to building materials. Other examples can be implemented without loss of generality. For example, indoor / outdoor spaces such as houses can be generalized to spaces including inside / outside spaces of vehicles such as automobiles, trains, ships, and airplanes. Building materials include flooring (board, carpet, stone, tile), inner wall (board, cloth (leather, synthetic leather, paper, synthetic resin, decorative sheet, etc.)) and outer wall (siding (board, stone, Tile materials, etc.) and ceiling materials (board, cloth (leather, leather, paper, synthetic resin, decorative sheets, etc.)), but vehicle interior materials (leather, leather, cloth, synthetic) Resin, decorative sheet, etc.) and car exterior materials, etc. can be generalized and described.

  FIG. 1 is a diagram illustrating a configuration of an image processing system. The image processing system includes an image processing apparatus 100, a virtual space image processing apparatus 300, a server 500, and a plurality of computers 702, 704, 706, and 708 (collectively referred to as “computer 700” in this specification). The server 500 includes a combination of material information (information relating to the texture of the material and a plurality of characteristics (color characteristics, shape characteristics, reflection characteristics; hereinafter also referred to as “elements”) from a plurality of computers 700. Various characteristics (color characteristics, shape characteristics, reflection characteristics) can be acquired. The server 500 can provide various data and / or services to the image processing apparatus 100 and the virtual space image processing apparatus 300.

  The plurality of computers 700 are, for example, computers installed in a system of a building material manufacturer or a wholesaler. Therefore, it is possible to simultaneously examine building materials handled by various manufacturers and wholesalers. Or the some computer 700 is a computer which the designer of a building material etc. have, for example. Therefore, it is possible to consider various combinations of colors, shapes, and reflection characteristics created not only by ready-made products but also by designers. The plurality of computers 700 may be, for example, computers in a company that provides the service according to the present invention. Therefore, data input to a computer in a company that provides a service according to the present invention can be used as material information and various characteristics.

  FIG. 2 shows an example of a virtual space displayed on a monitor (display device or the like) provided in the virtual space image processing apparatus 300. The information regarding the virtual space includes floor plan information such as walls, floors, ceilings, doors, windows, and lighting. The virtual space information is stored in the virtual space information storage unit 308 of the virtual space image processing device 300, and is used when the rendering unit 304 of the virtual space image processing device generates display data. The virtual space information may be created by the virtual space image processing apparatus 300 and stored in the virtual space information storage unit 308, or may be acquired from the server 500 and stored in the virtual space information storage unit 308.

  The virtual space in FIG. 2 includes various parts (walls 1 to 3, floor 1, ceiling 1, door 1, window 1, lights 1 and 2) that constitute a floor plan of the space. Virtual space information, which is information on the virtual space, includes information for identifying various parts and information for identifying the positions of the various parts in the virtual space. Information about light in the virtual space (light direction, color component (RGB information or spectral information representing intensity for each wavelength of light) and intensity) is defined as illumination information. The illumination information may include light distribution. By the information for identifying various parts, it is possible to identify the type of part (wall, floor, ceiling, door, etc.). Based on the information for identifying the position, the location of each part in the virtual space, the boundary between the other parts, the distance, and the relative positional relationship can be calculated. A method for selecting (determining) materials of various parts will be described later. When generating display data of the virtual space, information on materials of various parts (material information: including various characteristic information of materials), illumination information on light from each of the light sources 202, 204, 206 (lights 1 and 2 and the sun) Is considered.

  FIG. 3 is a diagram illustrating a user interface (material viewer) provided by the image processing apparatus 100 when selecting materials of various parts in the virtual space. For example, a home supplier (interior designer) uses the user interface shown in FIG. 3 to check the texture of materials of various parts (that is, freely combine various characteristics and change the material texture of the combination). You can select the material to apply. The virtual space image processing apparatus 300 generates and displays display information of the virtual space to which the material selected using the user interface of FIG. 3 is applied. Therefore, the housing supplier (interior designer) can experience the texture and appearance when the selected material is placed in the space.

  The image processing apparatus 100 and the virtual space image processing apparatus 300 share information regarding the virtual space. For example, as shown in FIG. 3, the user interface (material viewer) includes a parts list 402 that enables selection of various parts in the virtual space (FIG. 2). The user interface includes a list (material category list) 404 that allows selection of material categories applicable to the parts selected from the parts list.

  The user interface includes a list (element thumbnail list) 414 of thumbnails of various characteristics of the material classified into the category selected from the material category list. The element thumbnail list 414 includes a color characteristic thumbnail 416, a shape characteristic thumbnail 418, and a reflection characteristic thumbnail 420. That is, when various characteristics are collected from a plurality of computers 700, images representing the collected various characteristics are displayed as the color characteristic thumbnail 416, the shape characteristic thumbnail 418, and the reflection characteristic thumbnail 420. Further, the user interface includes a list of material thumbnails (material thumbnail list) 406 classified into a category selected from the material category list. That is, when material information (color characteristics, shape characteristics, and reflection characteristics are already combined) is collected from a plurality of computers 700, an image representing the collected material information is displayed in the material thumbnail list 406. It will be. The user interface also includes a texture viewer area 408 that displays an image of the material selected from the thumbnail list. That is, an image of a material composed of a combination of characteristics selected from each of the color characteristic thumbnail 416, the shape characteristic thumbnail 418, and the reflection characteristic thumbnail 420 is displayed. Alternatively, an image of the material selected by the material thumbnail 406 is displayed.

  Furthermore, the user interface includes a lighting environment list 412 that enables selection of lighting information (models 1, 2,...) Of various lighting environments modeled for the virtual space. The lighting environment list may include the lighting information of the real environment where the image processing apparatus 100 exists separately from the lighting information of the virtual space. The illumination environment list may include a summary of modeled illumination information (time zone, light source type / state, presence / absence of window, light distribution characteristic). The texture viewer area 408 displays material information (color characteristics, shape characteristics, reflection characteristics) and illumination information (light direction, light distribution, color component (RGB information or intensity for each wavelength of light) of the selected material. An image reflecting (spectral information) and intensity) is displayed. The texture viewer area 408 may be configured to occupy the entire display area of the display. As described below, the image processing apparatus 100 renders and displays an image of a material in consideration of the inclination of the image processing apparatus (tablet terminal) (the orientation of the display surface and the inclination with respect to the horizontal plane). Therefore, the viewer can observe the image of the material and experience the texture of the material as if he were holding the material.

  FIG. 4 is a functional block diagram of the server 500 of the image processing system according to the embodiment of the present invention. The server 500 is a computer that includes a processor and a memory.

  The processor may include a GPU and a coprocessor in addition to the CPU. The processor processes material information and various characteristics acquired from a plurality of computers 700. The processor includes an element collection unit 502, a data determination unit 504, a data conversion unit 506, and a production data generation unit 518.

  The memory may be either a magnetic drive such as an HDD or a semiconductor drive such as an SSD. The memory may be internal or external. The memory constitutes a material information storage unit 516. The material information storage unit 516 stores material information and various characteristics collected from the plurality of computers 700 via the communication unit 514, and the elements recorded in the element information storage unit 123 of the tablet terminal 100 via the communication unit 114. The combination information is recorded as additional information.

  The communication device may be one or more of a bus (for example, USB (Universal Serial Bus)) interface, a network interface card (NIC) for wired communication, and a wireless device for wireless communication for connection to an external device. The communication device constitutes the communication unit 514. The server 500 can receive material information and various characteristics from a plurality of computers 700 via the communication unit 514.

  FIG. 5 is a functional block diagram of a portable terminal which is an embodiment of the image processing apparatus according to the present invention. The portable terminal 100 in FIG. 5 is a tablet terminal, for example, and includes a display, a processor, a memory, and various sensors. The portable terminal 100 may include a communication device, a keyboard, a pointing device including a computer mouse, and an input device such as a microphone. The input device constitutes the input unit 112.

  The display device may be any display device suitable for displaying an image, and is not limited to this, but an electromagnetic induction type, capacitance type or pressure sensitive type touch display device is preferable. The display device constitutes the display unit 102. The electromagnetic induction type, capacitance type, or pressure sensitive touch display device constitutes the display unit 102 and the input unit 112.

  The processor may include a GPU and a coprocessor in addition to the CPU. The processor generates display data corresponding to an image displayed on the display unit. The processor constitutes a rendering unit (display data generation unit) 104 and an element combination unit 120.

  The memory may be either a magnetic drive such as an HDD or a semiconductor drive such as an SSD. The memory may be internal or external. The memory constitutes an illumination information storage unit 106, a material information storage unit 116, and an element information storage unit 123. The memory can also store information on the tilt of the tablet terminal (the orientation of the display surface and the tilt with respect to the horizontal plane) detected by the tilt detection unit 110.

  Various sensors are devices suitable for acquiring lighting information around the tablet terminal (ambient lighting information) (hereinafter referred to as ambient lighting information acquisition device) and devices suitable for detecting the tilt of the tablet terminal (tilt detection device). I just need it. For example, the ambient illumination information acquisition device can be one or more of a photometric device, an illuminance sensor, and a camera. The ambient lighting information acquisition device may be a built-in type or an external type. Further, for example, the tilt detection device can be one or more of a gyro sensor, an acceleration sensor, and a magnetic sensor. The ambient lighting information acquisition device constitutes the ambient lighting information acquisition unit 108 together with the processor or alone. The tilt detection device constitutes the tilt detection unit 110 together with the processor or alone. Further, the various sensors may include a device suitable for detecting the position of the viewer (for example, the viewer's eyes). Such a device can be one or more of an infrared sensor and a camera.

  The communication device may be one or more of a bus (for example, USB (Universal Serial Bus)) interface, a network interface card (NIC) for wired communication, and a wireless device for wireless communication for connection to an external device.

  For example, the camera as the ambient lighting information acquisition device can be a surrounding omnidirectional image camera or an omnidirectional camera, and an omnidirectional camera of the omnidirectional image obtained by photographing the entire periphery of the tablet terminal with such a camera. Color and brightness can be used as illumination information (ambient illumination information) in a real environment. Or, you can combine a reference image taken with a built-in or external camera with the camera placed on a horizontal surface, and an image of the entire periphery of the tablet device taken in multiple shots by changing the orientation of the camera. An all-around image can be generated (connected), and the color and brightness of the generated all-around image can be used as illumination information (ambient illumination information) in the real environment. If the dynamic range (latitude) of the image sensor included in the camera is narrower than the brightness range of the actual environment brightness distribution around the tablet device, a photographic technique (High Dynamic High dynamic range imaging (HDR) may be used, and light distribution characteristics may be obtained simply by capturing changes in brightness while moving the camera in front of the luminaire.

  For example, the position of the face (eyes) of the operator (viewer) of the tablet terminal is determined from an image taken by the built-in camera of the tablet terminal (or an external camera whose relative positional relationship with the tablet terminal is known). By specifying, the relative positional relationship between the display surface of the tablet terminal and the viewer can be specified. The relative positional relationship between the display surface of the tablet terminal and the viewer can be specified in real time by repeating shooting and specifying the position of the viewer's face (eyes) at short time intervals.

  The communication device constitutes the communication unit 114. The tablet terminal 100 may also acquire illumination information (illumination information modeled for the virtual space (space illumination information)) and material information from the outside (the virtual space image processing device 300 or the server 500) via the communication unit 114. it can. The illumination information acquired from the outside can be used as an alternative to the ambient illumination information (real space illumination information).

  The illumination information storage unit 106 stores the ambient illumination information acquired by the ambient illumination information acquisition unit 108 or the spatial illumination information acquired from the outside via the communication unit 114.

  The material information storage unit 116 stores material information acquired from the outside via the communication unit 114. The material information is information regarding the texture of the material. The material information is information indicating the direction of the surface of the pixel in the normal information (shape characteristic: equipment (floor material, wall material, ceiling material)) for each pixel (pixel). For example, the normal vector information of the pixel And / or depth information. The material information also includes RGB information (color characteristics: information indicating the color of each layer constituting the material, such as the background of the material and the coat layer) for each pixel. The material information also includes gloss information for each pixel (reflection characteristics: information indicating the gloss of the surface constituting the material, such as a coat layer provided on the surface of the material). As the gloss information that is the reflection characteristic, for example, information that is modeled by reflection using a bidirectional reflectance distribution function (BRDF) or a bidirectional scattering surface reflectance distribution function (BSSRDF) can be used. The modeled gloss information includes, for example, information composed of gloss intensity information and gloss sharpness information. At this time, the gloss information can further include gloss direction information in addition to gloss intensity information and gloss sharpness information. The texture of the material having anisotropic reflection can be expressed by the information on the direction of gloss. The information on the intensity of gloss may be information on the intensity of each pixel for each RGB. Display data is generated based on ambient illumination information (or spatial illumination information acquired from the outside) and material information. As a result, in addition to the color of the material, the texture of the material (glossiness, roughness, unevenness) is displayed as an image.

  The element information storage unit 123 stores information on results obtained by the element combination unit 120. That is, the element information storage unit 123 stores combination information indicating information element combinations (which color characteristics, which shape characteristics, and which reflection characteristics are combined). This information may be transferred to the communication unit 514 of the server 500 via the communication unit 114, recorded as additional information in the material information storage unit 516, and used in the process of manufacturing the material.

  The tablet terminal 100 includes a UI providing unit (not shown) that provides a user interface (UI) (for example, the material viewer of FIG. 3) for interaction with the viewer. As described above, the viewer can select a material to be displayed via the UI. That is, the tablet terminal 100 can receive selection of a material from the viewer via the UI and display the material on the display. The UI may include a hierarchical category menu (material category) that allows the viewer to select a material that the user wants to display. For example, the materials may be classified into categories such as wall materials, floor materials, and ceiling materials so that the viewer can select the materials according to the category hierarchy. In the subcategory, thumbnails of a plurality of materials or elements may be presented to allow the viewer to select materials or elements that the viewer wants to display.

  FIG. 6 is a functional block diagram of the virtual space image processing apparatus of the image processing system according to the embodiment of the present invention. The virtual space image processing apparatus 300 is a computer suitable for rendering and displaying a virtual space image, and includes a monitor, a processor, and a memory. The computer constituting the virtual space image processing apparatus 300 may include a communication device, a keyboard, a pointing device including a computer mouse, and an input device such as a microphone. The input device constitutes the input unit 312.

  The monitor may be a display device or the like suitable for image display in the virtual space, and constitutes the display unit 302.

  The processor may include a GPU and a coprocessor in addition to the CPU. The processor generates display data corresponding to an image displayed on the display unit. The processor constitutes a rendering unit (display data generation unit) 304. In addition, the processor constitutes an in-space viewer position detection unit 318 that detects the position of the viewer (for example, the viewer's eyes (viewpoint) or the camera) in the virtual space together with the input device or alone.

  The memory may be either a magnetic drive such as an HDD or a semiconductor drive such as an SSD. The memory may be internal or external. The memory constitutes a spatial illumination information storage unit 306, a spatial information storage unit 308, and a material information storage unit 316.

  The communication device may be one or more of a bus (for example, USB (Universal Serial Bus)) interface, a network interface card (NIC) for wired communication, and a wireless device for wireless communication for connection to an external device. The communication device constitutes the communication unit 314. The computer 300 constituting the virtual space image processing apparatus can send and receive space information, space illumination information, and material information to and from the outside (the portable terminal 100 or the server 500) via the communication unit 314. The spatial illumination information storage unit 306 stores spatial illumination information. The material information storage unit 316 stores material information. The spatial information storage unit 308 stores spatial information.

  The rendering unit (display data generation unit) 304 and the space insider position detection unit 318 of the virtual space image processing apparatus 300 may be mounted on the server 500. When these functions are implemented in the server 500, the virtual space image processing apparatus 300 can receive display data of the space rendered by the server 500 via the communication unit 314 and display it on the display unit 302. . Further, the rendering unit (display data generation unit) 104 of the portable terminal 100 may be mounted on the server 500. When these functions are implemented in the server 500, the portable terminal 100 can receive the display data of the material rendered by the server 500 via the communication unit 114 and display it on the display unit 102.

  FIG. 7 shows a space in which ambient lighting information is acquired by the ambient lighting information acquisition unit 108. In FIG. 7, the x-axis and the y-axis are axes orthogonal to each other in the horizontal plane, and the z-axis is an axis orthogonal to the horizontal plane. In the space of FIG. 7, a total of three light sources including two lamps 202 and 204 and the sun 206 are shown. With the intersection of the x-axis, y-axis, and z-axis as the position of the ambient illumination information acquisition unit 108, light from the two light sources 202 and 204 and the sun 206 is observed, and the direction of the light observed for each light source Light distribution, color components (RGB information or spectral information representing intensity for each wavelength of light) and intensity (luminance) are acquired as ambient illumination information in the space. The number of light sources is not limited to three. The light source is not limited to a light emitter, and may be a reflector that reflects light toward the ambient illumination information acquisition unit 108. Ambient illumination information (light direction, light distribution, color component (RGB information or spectral information representing intensity for each wavelength of light) and intensity) of all light sources observed in space may be acquired. Alternatively, among the illumination information in the modeled space, only the light direction, the light distribution information, and the intensity may be used, and the observed color component may be combined for the color component. The acquired ambient illumination information is stored in the illumination information storage unit 106.

  As an alternative to the ambient lighting information, the spatial lighting information can be acquired from the outside via the communication unit 114. Spatial lighting information and ambient lighting information are simply referred to as lighting information. Spatial lighting information includes information about lighting in a modeled space, such as a room with a south-facing window (a window through which sunlight is inserted) and four downlights, or a bedroom without a window and one electric lamp (externally in advance) Lighting information stored in the virtual space image processing apparatus 300 or the server 500). The modeled space may be one or more spaces in the model room floor plan of a building being sold / constructed.

  FIG. 8 shows a state in which the tablet terminal 100 is arranged in the space shown in FIG. FIG. 8 also shows the viewer's eyes 208 of the image displayed by the tablet terminal 100. The tablet terminal 100 is arranged at the intersection of the x-axis, the y-axis, and the z-axis so that the display surface faces upward and the display surface is parallel to the xy plane. The orientation of the eyes 208 may be the orientation relative to the normal of the display surface detected by the viewer detection unit 118, or may be the orientation predetermined for the normal of the display surface (not detected by the viewer detection unit 118). Good.

  In the state shown in FIG. 8, the rendering unit 104 generates material display data as if the surface of the material is superimposed on the display surface. When the generated display data is displayed on the display device, the viewer who has the tablet terminal 100 can observe the texture of the material as if he / she had the material with his / her hand. .

  The rendering unit 104 stores material information of the material to be rendered from the material information storage unit 116 (normal information and / or depth information (shape characteristic) for each pixel, RGB information (color characteristic), gloss intensity information, and gloss information. Sharpness information (and gloss directionality information) is read out, and illumination information (light direction, light distribution, color component (RGB information or light wavelength from one or more light sources) from the illumination information storage unit 106) Spectral information) and intensity (luminance)) are read out, and the color component and intensity (luminance) reflected in the orientation of the viewer in each pixel are calculated to generate display data. The generated display data is used, and an image is displayed on the display unit. In the state shown in FIG. 8, the color component and intensity (luminance) of light incident on the surface of the material from three light sources (two lamps 202 and 204 and the sun 206) and reflected in the direction of the eye 208 are calculated. .

  In the state shown in FIG. 9, the display surface of the tablet terminal 100 is inclined from the horizontal plane (xy plane). This inclination is such an inclination that the light from the electric lamp 202 does not enter the display surface. The inclination detection unit 110 detects this inclination, and the rendering unit 104 considers it when generating display data. That is, the rendering unit 104 calculates the display data by calculating the color component and intensity (luminance) of light incident on the surface of the material from two light sources (one lamp 204 and the sun 206) and reflected in the direction of the eye 208. Generate.

  Furthermore, in the state illustrated in FIG. 10, the display surface of the tablet terminal 100 is disposed in parallel to the z axis. This arrangement is an arrangement in which the lights of the two lamps 202 and 204 do not enter the display surface. The tilt detection unit 110 detects the tilt of the tablet terminal 100 at this time, and the rendering unit 104 considers it when generating display data. That is, the rendering unit 104 generates display data by calculating the color component and intensity (luminance) of light incident on the surface of the material from only one light source (sun 206) and reflected in the direction of the eye 208.

  As described with reference to FIGS. 8 to 10, the display data generated by the rendering unit 104 reflects the tilt of the tablet terminal 100 (the orientation of the display surface and the tilt relative to the horizontal plane) and the position of the eye (azimuth). Yes. Therefore, the viewer who has the tablet terminal 100 can feel the texture of the material by tilting the tablet terminal 100 or changing the position (orientation) of the eyes, as when observing the actual material. Can be observed.

  The rendering unit 104 can also implement a zoom function. As described above, the rendering unit 104 generates material display data as if the surface of the material is superimposed on the display surface. The distance between the display surface and the viewer holding the tablet terminal 100 is approximately the length of the viewer's arm and does not change significantly. Therefore, in response to a zoom-in instruction from the viewer received through the user interface (UI), the display data of the material is displayed as if the material is placed between the display surface and the viewer. Whether the display surface is placed between the material and the viewer (the material is placed behind the display surface) in response to a zoom-out indication that is generated or received via the UI As described above, it is useful to mount the zoom function for generating the display data of the material in the rendering unit 104. For example, the viewer holds the tablet terminal so that the display surface is perpendicular to the horizontal plane, and uses the zoom-out function to make the wall material 2 to 3 m away when making the material of the wall material. Assuming that it is possible to observe the texture of the material.

  FIG. 11 is a flowchart of processing of the server 500.

  In step S1101, the server 500 (element collection unit 502) can receive an element (at least one of color characteristics, shape characteristics, and reflection characteristics) from the plurality of computers 700. That is, the server 500 receives only color characteristics from a certain computer, receives only shape characteristics from another computer, receives only reflection characteristics from another computer, and the portable terminal 100 and the like combine these. An image of the material can be displayed based on the material information. Of course, the server 500 receives material information (that is, color characteristics, shape characteristics, and reflection characteristics are already combined) from one computer, and the portable terminal 100 or the like receives the material information based on the material information. The image of can also be displayed.

  In step S1103, the server 500 (data determination unit 504) can determine whether the material information or the element data received in step S1101 has a predetermined format. If the format is not a predetermined format, the process proceeds to step S1105. If the format is a predetermined format, the process ends (the received material information and element data are stored in the material information storage unit 516). . For example, the predetermined format of the color characteristic can be RGB information (color characteristic: information indicating the color of each layer constituting the material such as the background of the material and the coat layer) for each pixel. In addition, for example, the predetermined format of the shape characteristic is information indicating the orientation of the surface of the pixel in the normal line information (shape characteristic: equipment (floor material, wall material, ceiling material)) for each pixel (pixel). For example, pixel normal vector information) and / or depth information. Further, for example, the predetermined format of the reflection characteristics can be information on the intensity of gloss, information on the sharpness of the gloss, and information on the direction of gloss (reflection characteristics) for each pixel. The information on the intensity of gloss may be information on the intensity of each pixel for each RGB.

  In step S1105, the server 500 (data conversion unit 506) can convert the material information or element data received in step S1101 into data of a predetermined format. The converted data is stored in the material information storage unit 516. Therefore, various types of data collected from a plurality of computers 700 can be managed as common format data.

  Specifically, in the case of color characteristics, for example, the server 500 can receive data in the CMYK format (CMYK distribution) from the computer 700 and convert it to the RGB format (RGB distribution). Conversion from CMYK to RGB can be performed using an ICC profile.

  In the case of color information, the server 500 may have image information (plate information) composed of grayscale information and ink color spectral reflectance information. In this case, the data conversion unit 506 of the server 500 can convert color information obtained by combining grayscale information and ink color into RGB using a Neugebauer model or the like.

  In the case of shape characteristics, for example, the server 500 can receive information on the pattern shape and the plate depth of the embossed plate from the computer 700 and convert it into normal vector information for each pixel. For example, the information on the pattern shape and the plate depth of the embossed plate can be obtained by defining the depth information for each pixel by image data. Since the gray image signal can be expressed in 256 gradations from 0 to 255, 0 corresponds to the plate depth of 0, 255 corresponds to the plate depth of D μm (maximum plate depth), and the maximum plate depth D represents the embossed plate. Each can be specified as text data separately. Conversion from plate depth information to normal vector information can be performed as follows. When the difference in depth between adjacent pixels in the left-right (x-axis) direction and the up-down (y-axis) direction is determined for the pixel of interest, the inclination in the x-axis direction and y-axis direction of the pixel of interest Is derived. Since the derived inclination corresponds to the x-axis direction component and the y-axis direction component of the surface inclination of the pixel of interest, the surface inclination can be defined. Therefore, normal vector information can be determined.

  Further, the reflection characteristics are assumed to be a case where gloss is different for each pixel and a case where the gloss is uniform over the entire surface. When the gloss is different for each pixel, for example, the computer 700 can designate the gloss as an image. For example, an RGB image can be used to describe the intensity of gloss in the R channel, the sharpness of the gloss in the G channel, and the direction of gloss in the B channel. In the R channel and the G channel, 0 corresponds to the gloss strength and the sharpness of the gloss respectively, and 255 corresponds to the glossiness strength and the sharpness (spreading) of the glossy respectively in advance. Can be. In the B channel, 0 corresponds to a reference axis (for example, the x-axis and the y-axis that are orthogonal to each other on the material surface), and 255 is an angle rotated around an axis orthogonal to the material surface is 360 degrees. It can correspond to. Based on the RGB image received from the computer 700, the server 500 can determine the gloss intensity information and the gloss sharpness information for each pixel. In the case of uniform gloss over the entire surface, for example, a mathematical expression (uniform distribution formula) representing reflection characteristics is defined in advance. Then, the server 500 can receive from the computer 700 numerical values for designating the intensity of gloss and the sharpness of gloss, which are parameters of the mathematical formula. The server 500 can generate an RGB image for designating gloss as an image based on the received numerical value.

  FIG. 12 is a flowchart of processing performed by the image processing apparatus, and is an example of a processing flow executed by the portable terminal 100 described above.

  In step S601, the portable terminal 100 (rendering unit 104) acquires ambient lighting information. The illumination information acquired by the ambient illumination information acquisition unit 108 or the illumination information stored in the ambient illumination information storage unit 106 in the real environment where the portable terminal is arranged is acquired. Alternatively, the external illumination information in the modeled space acquired via the communication unit 114 or the external illumination information acquired via the communication unit 114 and stored in the ambient illumination information storage unit 106 is acquired. The viewer may select from illumination information stored in the ambient illumination information storage unit 106 in advance or external illumination information in the modeled space acquired via the communication unit 114.

  In step S603, the portable terminal 100 (element combination unit 120) determines material information by combining a plurality of elements received by the server 500 from a plurality of computers 700. For example, the portable terminal 100 can determine material information by combining the characteristics selected in each of the color characteristic thumbnail 416, the shape characteristic thumbnail 418, and the reflection characteristic thumbnail 420 of the user interface of FIG. The combination information indicating the combination of the element information (which color characteristic, which shape characteristic, and which reflection characteristic) constituting the determined material information is stored in the element information storage unit 123 and / or the server via the communication unit 114. 500 is transferred to the communication unit 514 and recorded as additional information in the material information storage unit 516. For example, an identifier for identifying a new material composed of a combination of a plurality of elements may be given by the server 500 and stored in the additional information. For example, the association information is stored in the element information storage unit 123 and / or the material information storage unit 516 together with the identifier. The storage of the association information and the additional information may be executed when it is determined in step S613 that it is determined that the material displayed on the display unit 102 is applied to the virtual space part.

  In step S605, the portable terminal 100 (rendering unit 104) acquires material information (color characteristics, shape characteristics, reflection characteristics) of the material to be displayed. Material information acquired from the outside via the communication unit 114 and stored in the material information storage unit 116 (that is, the material information determined in step S603) is acquired. For example, the color characteristic is RGB information for each pixel of the material, the shape characteristic is normal vector information and / or depth information for each pixel of the material, and the reflection characteristic is information on the intensity of gloss and gloss of each pixel. Information on sharpness and directionality of gloss.

  In step S607, the portable terminal 100 (rendering unit 104) identifies the relative positional relationship between the illumination, the display surface, and the viewer. The portable terminal 100 (rendering unit 104) determines the relative positional relationship between the illumination, the display surface, and the viewer, the orientation of light included in the illumination information, at least one of the orientation of the display surface and the orientation of the viewer's eyes. It can specify by calculating using. The tilt of the portable terminal and the orientation of the viewer's eyes are detected by the tilt detection unit 110 and the viewer detection unit 118, respectively, and are held in a memory or the like so that the rendering unit 104 can access them.

  In step S609, the portable terminal 100 (rendering unit 104) determines the material information based on the acquired illumination information, the calculated relative positional relationship, and the acquired material information (color characteristics, shape characteristics, reflection characteristics). Render the display data.

  In step S611, the display unit 102 displays (redisplays) the material image using the display data.

  In step S613, it is determined whether or not the material displayed as an image on the display unit 102 is selected to be applied to parts in the virtual space. If the application of the material is not selected, the process proceeds to step S618.

  If the application of the material is selected, in step S615, the identifier of the part of the virtual space and the identifier of the selected material are transmitted. The transmitted identifier of the virtual space part and the selected material identifier are received by the virtual space image processing apparatus 300 or the server 500.

  In step S618, it is determined whether termination of the program has been instructed. If the program is instructed to end, the process flow ends. If the end of the program is not instructed, the process returns to step S607.

  Note that step S601 may be executed after step S605. In particular, when using illumination information acquired in a real environment, step S601 may be included in a loop from step S607 to S611 so that the illumination information is acquired again each time the rendering unit 104 renders display information. .

  FIG. 13 is a flowchart of processing of the virtual space image processing apparatus.

  In step S700, the virtual space image processing device 300 (rendering unit 304) acquires virtual space information. Information on the space stored in advance in the space information storage unit 308 (information on the space including various parts constituting the space layout) is acquired. The spatial information may be spatial information acquired from the server 500 or the portable terminal 100 via the communication unit 314 and stored in the spatial information storage unit 308.

  In step S701, the virtual space image processing device 300 (rendering unit 304) acquires virtual space illumination information. The illumination information (spatial illumination information) of the modeled space stored in advance in the spatial illumination information storage unit 306 is acquired. The spatial illumination information may be spatial illumination information acquired from the server 500 or the portable terminal 100 via the communication unit 314 and stored in the spatial illumination information storage unit 306. The ambient illumination information acquired by the ambient illumination information acquisition unit 108 of the portable terminal 100 and stored in the illumination information storage unit 106 may be used as the spatial illumination information.

  In step S703, the virtual space image processing apparatus 300 (rendering unit 304) receives the part identifier and the material identifier to be applied to the part from the portable terminal 100, and responds to the material information (color Characteristics, shape characteristics, reflection characteristics). Material information stored in advance in the material information storage unit 316 is acquired. The material information can be material information acquired from the server 500 or the portable terminal 100 via the communication unit 314 and stored in the material information storage unit 316. This is information indicating material information, the above-described color characteristics, shape characteristics, and reflection characteristics.

  In step S705, the virtual space image processing device 300 (viewer position detection unit 318) determines the position of the viewer's eyes (viewpoint or camera position) in the virtual space.

  In step S707, the virtual space image processing device 300 (rendering unit 304) renders display data of the space (virtual space) in which the identified material is arranged. The rendering of the display data of the space is based on the acquired illumination information and material information, and the determined position of the viewer's eyes in the virtual space.

  In step S709, the virtual space image processing device 300 (display unit 302) displays (redisplays) an image of the space (virtual space) using the display data.

  In step S711, whether or not the viewer's eyes (viewpoint) in the virtual space has been changed (whether the position of the viewpoint in the virtual space has been changed, whether the virtual space has been rotated, or has been scaled). decide. If the viewer's eyes (viewpoint) are changed, the process proceeds to step S713. If the viewer's eyes (viewpoint) are not changed, the process proceeds to step S714. In steps S713 and S714, it is determined whether an instruction to end the program is given.

  If the end of the program is instructed in step S713, the process flow is ended. If the end of the program is not instructed, the process returns to step S705, and rendering of the display data in the space (virtual space) is performed again. Is displayed again (step S709). If the end of the program is instructed in step S714, the process flow is ended. If the end of the program is not instructed, step S711 is repeated. Alternatively, if steps S711 and S714 are inserted between steps S705 and S707 and step S705 is repeated until the viewer's eyes (viewpoint) are changed, the viewer's eyes (viewpoint) are changed. The process may proceed to step 713 after executing steps S707 to S709.

  By any means, every time the viewer's eyes (viewpoint) in the virtual space are changed by the virtual space image processing device 300 (input unit 312), the rendering is performed at a high speed and the image is displayed again. The effect can be given to the viewer as if it were moving freely and smoothly.

  FIG. 14 is a flowchart of processing for generating manufacturing data of the image processing system according to the embodiment of the present invention. In the present invention, the server 500 receives information as a result of combining elements from the portable terminal 100, and is necessary for manufacturing a material (that is, manufacturing a real wallpaper or the like) based on the information. Data can be generated.

  In step S801, the manufacturing data generation unit 518 of the server 500 acquires material information (color characteristics, shape characteristics, reflection characteristics) of a material to be manufactured. Material information (element information, that is, combination information indicating which color characteristic, which shape characteristic, and which reflection characteristic are combined) stored in the material information storage unit 516 is acquired. For example, the color characteristics are RGB information for each pixel of the material, the shape characteristics are normal vector information for each pixel of the material, and the reflection characteristics are gloss intensity information, gloss sharpness information and gloss for each pixel. It is information of directionality. Specifically, the manufacturing data generation unit 518 acquires material information designated via the user interface (element thumbnail 414) of FIG.

  In step S803, the manufacturing data generation unit 518 converts the material information acquired in step S801 into manufacturing data. That is, color characteristics, shape characteristics, and reflection characteristics are converted into data used in the material manufacturing process.

  Specifically, in the case of color characteristics, for example, the manufacturing data generation unit 518 can convert the color characteristics (RGB information for each pixel of the material) into data in CMYK format. Conversion from RGB to CMYK can be performed using an ICC profile. Based on the data in the CMYK format, the material is output by a manufacturing apparatus (such as a printing machine). In addition, when color information is stored as image information (plate information) consisting of grayscale information and ink color information (CIELAB value and spectral reflectance), CCM (Computer Color Matching) technology is used from the ink color information. Thus, the ink can be made.

  In the case of shape characteristics, for example, the manufacturing data generation unit 518 uses the shape characteristics (normal vector information for each pixel of the material) as processing data for the manufacturing apparatus (cutting coordinates, etching mask pattern, three-dimensional coordinates, etc.) Can be converted to First, the inclination of the surface of each pixel is determined from the normal vector information of each pixel. That is, the left-right (x-axis) direction component and the up-down (y-axis) direction component of the surface tilt in each pixel are determined. These components correspond to the inclination of each pixel in the x-axis direction and the y-axis direction. Next, based on the inclinations in the x-axis direction and the y-axis direction, a difference in depth from adjacent pixels in each of the x-axis direction and the y-axis direction is obtained. And the uneven | corrugated information of a raw material can be determined.

  In the case of reflection characteristics, for example, the manufacturing data generation unit 518 applies the reflection characteristics (gloss intensity information, gloss sharpness information, and gloss direction information) for each pixel of the material by the manufacturing apparatus. Can be converted into information for specifying the varnish to be used. Specifically, printing in which the blending ratio is changed stepwise in advance for each combination of the type of mat varnish and the type of gloss varnish (that is, for each combination of one type of mat varnish and one type of gloss varnish). Create a sample. Then, the gloss characteristics of the print sample are measured, and a correspondence table of mat varnish type and gloss varnish type, varnish compounding ratio, and gloss characteristics is prepared. Further, this gloss characteristic is associated with the reflection characteristic. Therefore, the manufacturing data generation unit 518 can determine information for designating the varnish (that is, the type of mat varnish and the type of gloss varnish, the blending ratio of varnish) based on the reflection characteristics.

  In step S805, the manufacturing data generation unit 518 can transmit the manufacturing data generated in step S803 to the manufacturing apparatus.

  As described above, according to the present invention, a plurality of elements collected from a plurality of computers can be freely combined to display an image of the texture of the material in accordance with changes in the observation environment and the orientation of the material, It is possible to provide an image processing system, method, and program that enable a pseudo-experience of texture and appearance when the material is arranged in a space.

  In the description of the above embodiment, the configuration in which the image processing system includes the portable terminal (tablet terminal) and the virtual space image processing terminal (computer) has been described. However, the functional unit of the virtual space image processing terminal is the portable terminal. Or the functional unit of the portable terminal can be mounted on a virtual space image processing terminal to implement the present invention. In addition, the server can be included in the configuration of the image processing system, and some functions of the portable terminal or the virtual space image processing terminal can be mounted on the server to implement the present invention.

100 portable terminal 102 display unit (pressure-sensitive display)
104, 304 Rendering unit (display data generating unit)
106,306 Illumination information storage unit 108 Ambient illumination information acquisition unit (photometric device, illuminance sensor, camera)
110 Tilt detection unit (gyro sensor, acceleration sensor, magnetic sensor)
112, 312 Input unit (keyboard, keypad, mouse, pointing device, microphone)
114, 314, 514 Communication unit (network IF card (NFC), wireless device)
116, 316, 516 Material information storage unit 118 Audience detection unit (infrared sensor, camera)
120 Element combination part 123 Element information storage part 202,204 Electric light 206 Sun 208 Eye 300 Computer 302 Display part (monitor)
308 Spatial information storage unit 318 Space insider position detection unit 500 Server 502 Element collection unit 504 Data determination unit 506 Data conversion unit 518 Manufacturing data generation unit 702, 704, 706, 708 Computer

Claims (27)

An image processing system connected to a plurality of computers, comprising: an element collection means; an element combination means; a first display means for displaying an image of a space; and a first rendering means for rendering display data of the space And second display means for displaying an image of the material applied to the space, and second rendering means for rendering display data of the image of the material,
The element collecting means is configured to collect at least one of a color characteristic, a shape characteristic, and a reflection characteristic for each pixel of a material image from each of the plurality of computers.
The element combination means is configured to determine material information regarding the texture of the material by combining the color characteristics, shape characteristics, and reflection characteristics collected by the element collection means,
The second rendering means is based on illumination information, material information on the texture of the material determined by the element combination means, and illumination, a display surface of the second display means, and a relative positional relationship between the viewers. Configured to render display data of the material,
The second display means is configured to display an image of the material using the rendered display data of the material,
In response to an instruction to adapt the material to the space, the first rendering means displays the space display data to which the material is applied based on the spatial illumination information of the space and the material information on the texture of the material. Is configured to render
The image processing system configured to display the image of the space using the rendered display data of the space. Further comprising tilt detection means configured to detect the orientation of the display surface of the second display means;
The second rendering means is configured to render display data of the material based on illumination information, material information on the texture of the material, and an orientation of a display surface of the second display means. 2. The image processing system according to 1.   The apparatus further comprises ambient illumination information acquisition means configured to acquire ambient illumination information of the second display means, and the illumination information used when the display data of the material is rendered is the ambient illumination information acquisition The image processing system according to claim 1, wherein the ambient illumination information is acquired by a means.   The image processing system according to claim 1, wherein the illumination information used when the display data of the material is rendered is the spatial illumination information used when rendering the display data of the space.   The apparatus further comprises ambient illumination information acquisition means configured to acquire ambient illumination information of the second display means, and the spatial illumination information used when rendering the display data of the space is the ambient illumination information acquisition The image processing system according to claim 1, wherein the ambient illumination information is acquired by a means. Further comprising viewer detection means for detecting the orientation of the viewer of the image of the material,
The second rendering unit is configured to display the material display data based on the illumination information, material information on the texture of the material, the orientation of the display surface of the second display unit, and the orientation of the viewer of the image of the material. The image processing system according to claim 2, wherein the image processing system is configured to render an image. The lighting information includes at least a light direction, a color component, and an intensity,
The image processing system according to claim 1, wherein the material information includes color characteristics, shape characteristics, and reflection characteristics for each pixel of the image of the material. The image processing system includes a spatial image processing terminal and a portable terminal,
The spatial image processing terminal includes the first rendering means and the first display means,
The image processing system according to claim 1, wherein the portable terminal includes the second rendering unit and the second display unit. The image processing system further includes manufacturing data generation means,
The manufacturing data generating means is configured to generate and output manufacturing data of the material based on the color characteristics, shape characteristics and reflection characteristics of the material information of the materials combined by the element combining means. The image processing system according to claim 1. Element collection means, element combination means, first display means for displaying an image of the space, first rendering means for rendering display data of the space, and an image of a material applied to the space are displayed. An image processing method in an image processing system comprising second display means and second rendering means for rendering display data of an image of the material, wherein the image processing system is connected to a plurality of computers. And said method is
The element collecting means collects from each of the plurality of computers at least one of a color characteristic, a shape characteristic, and a reflection characteristic for each pixel of a material image;
The element combination means determines the material information on the texture of the material by combining the color characteristics, shape characteristics and reflection characteristics collected by the element collection means;
The second rendering means is based on illumination information, material information on the texture of the material determined by the element combination means, and illumination, a display surface of the second display means, and a relative positional relationship between the viewers. Rendering the display data of the material;
The second display means displays the image of the material using the rendered display data of the material;
In response to an instruction to adapt the material to the space, the first rendering unit renders display data of the space to which the material is applied based on spatial illumination information and material information on the texture of the material. And
The first display means includes displaying an image of the space using the rendered display data of the space. The image processing system further includes tilt detection means,
The method further includes the tilt detection means detecting the orientation of the display surface of the display means,
The second rendering means renders the display data of the material by the second rendering means of the display means detected by the illumination information, material information relating to the texture of the material, and the inclination detecting means. The method of claim 10, comprising rendering display data of the material based on an orientation of a display surface. The image processing system further comprises ambient lighting information acquisition means,
The method further includes the ambient lighting information acquisition means acquiring ambient lighting information of the second display means,
The method according to claim 10 or 11, wherein the illumination information used when display data of the material is rendered is the ambient illumination information.   The method according to claim 10 or 11, wherein the illumination information used when the display data of the material is rendered is the spatial illumination information used when rendering the display data of the space. The image processing system further comprises ambient lighting information acquisition means,
The method further includes the ambient lighting information acquisition means acquiring ambient lighting information of the second display means,
Rendering the display data of the space to which the material is applied by the first rendering means is the ambient light acquired by the ambient light information acquiring means in response to an instruction to adapt the material to the space. The method according to claim 10, comprising rendering the display data of the space to which the material is applied based on the information and material information regarding the texture of the material. The image processing system further comprises a viewer detection means,
The method further includes the viewer detecting means detecting the orientation of the viewer of the image of the material,
The second rendering means renders the display data of the material because the second rendering means includes the illumination information, material information regarding the texture of the material, the orientation of the display surface of the second display means, and 15. A method according to any of claims 11 to 14, comprising rendering display data of the material based on a viewer orientation of the image of the material. The lighting information includes at least a light direction, a color component, and an intensity,
The method according to claim 10, wherein the material information includes color characteristics, shape characteristics, and reflection characteristics for each pixel of the image of the material. The image processing system includes a spatial image processing terminal and a portable terminal,
The spatial image processing terminal includes the first rendering means and the first display means,
The method according to claim 10, wherein the portable terminal includes the second rendering unit and the second display unit. The image processing system further comprises manufacturing data generation means,
In the method, the manufacturing data generation means generates and outputs the manufacturing data of the material based on the color characteristics, shape characteristics, and reflection characteristics of the material information of the materials combined by the element combination means. 18. A method according to any of claims 10 to 17, comprising. Element collection means, element combination means, first display means for displaying an image of the space, first rendering means for rendering display data of the space, and an image of a material applied to the space are displayed. A program executed by a computer system comprising second display means and second rendering means for rendering display data of an image of the material, wherein the computer system is connected to a plurality of computers. The program is
The element collecting means collects at least one of color characteristics, shape characteristics, and reflection characteristics for each pixel of a material image from each of the plurality of computers,
The element combination means is combined with the color characteristics, shape characteristics and reflection characteristics collected by the element collection means to determine material information regarding the texture of the material,
Based on the lighting information, the material information on the texture of the material determined by the element combination means, and the lighting, the display surface of the second display means, and the relative positional relationship of the viewer. , Render the display data of the material,
Causing the second display means to display an image of the material using the rendered display data;
In response to an instruction to adapt the material to the space, the first rendering means renders display data of the space to which the material is applied based on spatial illumination information and material information on the texture of the material. ,
The program which makes the said 1st display means display the image of the said space using the display data of the said rendered space. The computer system further includes tilt detection means,
The program causes the tilt detection means to detect the orientation of the display surface of the second display means,
Rendering the display data of the material on the second rendering means based on the illumination information, material information on the texture of the material, and the orientation of the display surface of the second display means detected by the inclination detecting means The program according to claim 19. The computer system further comprises ambient lighting information acquisition means,
The program causes the ambient lighting information acquisition unit to acquire ambient lighting information of the second display unit,
Based on the ambient lighting information, the material information, the illumination, the display surface of the second display means, and the relative positional relationship of the viewer, the display data of the material is sent to the second rendering means. The program according to claim 19 or 20, wherein the program is rendered.   21. The program according to claim 19, wherein the illumination information used when rendering the display data of the material is the spatial illumination information used when rendering the display data of the space. The computer system further comprises ambient lighting information acquisition means,
The program causes the ambient lighting information acquisition unit to acquire ambient lighting information of the second display unit,
Rendering the display data of the space to which the material is applied to the first rendering means is the ambient light acquired by the ambient light information acquiring means in response to an instruction to adapt the material to the space. 21. The program according to claim 19 or 20, comprising rendering the display data of the space to which the material is applied based on the information and material information on the texture of the material. The computer system further comprises spectator detection means,
The program causes the viewer detection means to detect the orientation of the viewer of the image of the material,
Based on the illumination information, the material information on the texture of the material, the orientation of the display surface of the second display means, and the orientation of the viewer of the image of the material, the display data of the material is sent to the second rendering means. The program according to any one of claims 20 to 23, wherein the program is rendered. The lighting information includes at least a light direction, a color component, and an intensity,
The program according to any one of claims 19 to 24, wherein the material information includes color characteristics, shape characteristics, and reflection characteristics for each pixel of the image of the material. The computer system includes a spatial image processing terminal and a portable terminal,
The spatial image processing terminal includes the first rendering means and the first display means,
26. The program according to claim 19, wherein the portable terminal includes the second rendering unit and the second display unit. The computer system further comprises manufacturing data generation means,
The program causes the manufacturing data generation means to generate and output manufacturing data of the material based on the color characteristics, shape characteristics, and reflection characteristics of the material information of the materials combined by the element combination means. Item 27. The program according to any one of Items 19 to 26.