JP5843359B2 - Information processing system, server device, terminal device, information processing device, information processing method, and program - Google Patents

Description

  The present invention relates to an information processing system or the like that outputs a surface stress of a processed product processed using a mold.

  As a conventional information processing system, an electric device, a detector for detecting a device state of the electric device, and first data for converting detection data detected by the detector into a communication signal and transmitting it to a site area A second communication signal converter that converts a communication signal received from the first communication signal converter into detection data before conversion by the first communication signal converter; And a diagnosis / analysis program for analyzing the device state from the detection data converted by the second communication signal converter, and a maintenance database for accumulating data and diagnosis results necessary for the analysis by the diagnosis / analysis program A device having a maintenance tool and a display for displaying a diagnosis result analyzed by the maintenance tool has been known (see, for example, Patent Document 1).

Japanese Patent Laid-Open No. 2003-69731 (first page, FIG. 1 etc.)

  However, in the conventional information processing system, there is a problem in that it is impossible to output information on the stress of the processed product for an image obtained by photographing the processed product processed using the mold.

  The information processing system of the present invention is an information processing system having a server device and a terminal device, and the server device processes mold identification information which is mold identification information and a member into a shape corresponding to the mold. One or more point information which is information indicating one or more points on the surface of the processed member, and information obtained by performing a predesignated analysis on the processed member. And point analysis information in which one or more pieces of point analysis information having one or more pieces of analysis information, which is information indicating an analysis result of each point indicated by one or more pieces of point information on the surface of the processed member, are stored. 1 of storage part, server receiving part which receives metal mold | die identification information from a terminal device, 1 or more point information corresponding to the metal mold | die identification information which the server receiving part received, and analysis information corresponding to each point information From the point analysis information storage unit An acquisition unit to be obtained, and a server transmission unit that transmits one or more sets of point information and analysis information acquired by the acquisition unit to the terminal device. The terminal device is obtained by processing a member using a mold. An imaging unit that acquires one or more captured images that are images of the processed product, a reception unit that receives mold identification information of a mold used for processing the processed product captured by the imaging unit, and a reception unit Using a terminal transmission unit that transmits the mold identification information received by the server device to the server device, a terminal reception unit that receives one or more sets of point information and analysis information from the server device, and one or more captured images. An imaging processed product shape data acquisition unit that acquires imaging processed product shape data, which is three-dimensional data indicating the shape of the processed product, and one of the captured images used to acquire the imaging processed product shape data For each of the above, the image processing product shape data Information indicating a setting for obtaining a projection that is obtained by projecting the shape indicated by the image, and a projection diagram in which the shape indicated by the imaged workpiece shape data in the projection diagram is the same as the shape of the workpiece in the captured image. A projection setting information acquisition unit that acquires projection setting information that is information indicating settings to obtain, and an analysis processed product that is three-dimensional data indicating the shape corresponding to the mold indicated by the mold identification information received by the reception unit Analytical processed product shape data acquisition unit that acquires shape data, a conversion unit that performs coordinate conversion of the analytical processed product shape data so that the analytical processed product shape data overlaps with the imaging processed product shape data, and analytical processing in which the conversion unit performs coordinate conversion Using the product shape data, the analysis information corresponding to point information indicating one or more points on the surface of the three-dimensional shape indicated by the analysis processed product shape data, and one of the projection setting information, the analysis processed product shape data indicates An analysis image generation unit that generates an analysis image, which is an image of a projection diagram showing information on the analysis result indicated by the analysis information on the surface of the shape, according to the setting indicated by the projection setting information, and a projection used by the analysis image generation unit An information processing system including an output unit that outputs an image obtained by superimposing a captured image corresponding to setting information and an analysis image.

  With this configuration, it is possible to output information related to the analysis result of the processed product with respect to an image obtained by capturing the processed product processed using the mold.

  In the information processing system of the present invention, in the information processing system, the point analysis information further includes member identification information that is information for identifying a member to be processed using the mold indicated by the mold identification information. The server receiving unit receives a set of mold identification information and member identification information from the terminal device, and the acquisition unit is one or more pieces of point information corresponding to the mold identification information and the member identification information received by the server receiving unit. And one or more sets of analysis information corresponding to each point information is acquired from the point analysis information storage unit, the reception unit further receives member identification information of the member used for processing, and the terminal transmission unit is It is the information processing system which transmits the metal mold | die identification information and member identification information which the reception part received to the said server apparatus.

  With this configuration, it is possible to output information related to the analysis result corresponding to the member used in the processed product, for an image obtained by photographing the processed product processed using the mold.

  In the information processing system of the present invention, in the information processing system, the imaging processed product shape data acquisition unit acquires imaging processed product shape data each time the imaging unit acquires one captured image, and sets projection setting information. The acquisition unit acquires projection setting information every time the imaging unit acquires one captured image, and the analysis image generation unit analyzes using the projection setting information acquired by the projection setting information acquisition unit for the one captured image. An image is generated, and the output unit is an information processing system that outputs an image in which one captured image and an analysis image are superimposed each time the image capturing unit acquires one captured image.

  With this configuration, each time a captured image is captured, information regarding the analysis result of the processed product can be output to the captured image.

  Further, the information processing apparatus according to the present invention includes mold identification information that is identification information of a mold and one or more points on the surface of the processed member when the member is processed into a shape corresponding to the mold. One or more point information that is information to be displayed and information obtained by performing a predesignated analysis on the processed member, wherein one or more point information on the surface of the processed member is Obtained by processing a member using a die, a point analysis information storage unit that stores one or more point analysis information having one or more pieces of analysis information that is information indicating an analysis result for each point shown An imaging unit that acquires one or more captured images that are images of a processed product, a reception unit that receives mold identification information of a mold used for processing the processed product captured by the imaging unit, and a reception unit One or more point information corresponding to the mold identification information and each point information This is three-dimensional data indicating the shape of a processed product that has been imaged using an acquisition unit that acquires one or more sets of corresponding analysis information from a point analysis information storage unit and one or more captured images. The shape indicated by the imaging processed product shape data is projected for each of one or more of the captured image used when acquiring the imaging processed product shape data and the imaging processed product shape data acquisition unit that acquires the imaging processed product shape data. Information indicating settings for obtaining a projection view, and information indicating settings for obtaining a projection view in which the shape indicated by the imaged workpiece shape data in the projection view is the same as the shape of the work piece in the captured image Projection setting information acquisition unit for acquiring projection setting information and analysis processing for acquiring analysis product shape data that is three-dimensional data indicating the shape corresponding to the mold indicated by the mold identification information received by the reception unit A shape data acquisition unit, a conversion unit that converts the analysis processed product shape data into coordinates so that the analysis processed product shape data overlaps the imaging processed product shape data, an analysis processed product shape data that is converted by the conversion unit, and an analysis processed product Using the analysis information corresponding to point information indicating one or more points on the surface of the three-dimensional shape indicated by the shape data and one of the projection setting information, the analysis information is indicated on the surface of the shape indicated by the analysis processed product shape data. An analysis image generation unit that generates an analysis image that is an image of a projection diagram showing information related to the analysis result according to the setting indicated by the projection setting information, and a captured image corresponding to the projection setting information used by the analysis image generation unit, And an output unit that outputs an image obtained by superimposing the analysis image.

  With this configuration, it is possible to output information on the analysis result of the processed product with respect to an image obtained by photographing the processed product processed using the mold.

  According to the information processing system or the like according to the present invention, it is possible to output information related to the analysis result of a processed product for an image obtained by photographing the processed product processed using a mold.

The block diagram of the information processing system in Embodiment 1 of this invention A flowchart for explaining the operation of the server device of the information processing system A flowchart for explaining the operation of the terminal device of the information processing system The figure which shows the conceptual diagram of the same information processing system The figure which shows the point management information of the information processing system The figure which shows the point analysis management information of the same information processing system The figure which shows a press molding processing machine for explaining operation of the information processing system (Drawing 7 (a)), and the figure which shows the member processed by press forming (Drawing 7 (b)) The figure which shows an example of the terminal device of the information processing system The figure which shows the point analysis acquisition information of the same information processing system Schematic diagram illustrating a state in which a processed product obtained by processing a member with a mold is photographed from different directions for explaining the information processing system (FIGS. 10A to 10D) The figure which shows the projection setting information of the information processing system The figure which shows the figure for demonstrating the process of the coordinate transformation of the information processing system (FIG. 12 (a)-FIG.12 (f)) The figure which shows an example of the analysis image of the information processing system The figure which shows the output example of the same information processing system FIG. 3 is a block diagram showing the configuration of the information processing apparatus according to the second embodiment. Flow chart for explaining the operation of the information processing apparatus Schematic diagram showing an example of the appearance of a computer that implements an information processing system and an information processing apparatus according to each embodiment of the present invention The figure which shows the internal configuration of the computer system The block diagram of the information processing system in Embodiment 3 of this invention A flowchart for explaining the operation of the server device of the information processing system

  Hereinafter, embodiments of an information processing system and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.

(Embodiment 1)
FIG. 1 is a block diagram of an information processing system 1 in the present embodiment.

  The information processing system 1 includes a server device 11 and a terminal device 12. The server device 11 and the terminal device 12 are communicably connected via a network or the like. Here, a case where there is one terminal device 12 will be described as an example. However, in the present invention, a plurality of terminal devices 12 may be provided.

  The server device 11 includes a point analysis information storage unit 110, a point analysis information acquisition unit 111, a server reception unit 112, an acquisition unit 113, and a server transmission unit 114.

  The terminal device 12 includes an imaging unit 121, a captured image storage unit 122, a reception unit 123, a terminal transmission unit 124, a terminal reception unit 125, an imaging processed product shape data acquisition unit 126, a projection setting information acquisition unit 127, and an analysis processed product shape data. An acquisition unit 128, a conversion unit 129, an analysis image generation unit 130, and an output unit 131 are provided.

  The point analysis information storage unit 110 stores one or more point analysis information. The point analysis information includes mold identification information, member identification information, one or more point information, and analysis information corresponding to each of the one or more point information.

  The mold identification information is mold identification information. The mold is, for example, a mold for manufacturing a metal or resin part by processing such as plastic processing such as press processing or molding processing such as injection molding. For example, one mold used for press working or the like is usually composed of a set of a concave member and a convex member. The mold material may be any material such as metal or ceramic. The mold identification information may be, for example, a code assigned to the mold or the name of the mold. In addition, the mold identification information may be information that can identify the mold as a result. For example, one mold is associated with the name or code of a processed product processed by the mold on a one-to-one basis. In this case, the mold identification information may be identification information such as a name or a code of a processed product processed with one mold.

  Member identification information is information which identifies the member processed using a metal mold | die. The member is an object to be processed, for example, and is, for example, a plate material such as metal, a sheet of resin, a pellet of resin, or the like. The member should just be a thing which can be processed using a metal mold | die. Here, the member may be considered as a material to be processed. Even if they are made of the same material, they may be considered to be different members when the thickness or the like is different. In particular, when the thickness of the member affects the stress of the processed product obtained by processing, it is preferable to consider the different members. The member identification information may be any information as long as the member can be identified. For example, the member identification information may be a code or the like assigned to the member, or the material or thickness of the member. The name etc. which combined information may be sufficient.

  The one or more point information is information indicating one or more points on the surface of the processed member when the member is processed into a shape corresponding to the mold. The shape described here is a three-dimensional shape. The processed member described here is specifically a processed product obtained by processing. However, the processed product here is, for example, when a member is processed according to mold design information (for example, information such as design values or three-dimensional data indicating the shape of a mold created by CAD or the like). A virtual (or ideal) processed product that is considered to be obtained is preferable for analysis by simulation or the like. The virtual processed product is represented by, for example, three-dimensional data. The point of the surface of the processed member is, for example, the shape of the member processed by the mold expressed by a three-dimensional model using a mesh such as a polygon mesh or a line such as a spline curve that defines a free-form surface. In this case, it is a node of a mesh or a line representing the surface of the processed member. A node is an intersection of lines constituting a mesh, a spline curve or the like, or a point where these lines are connected. Also, the anchor point of a line may be considered as a node. The one or more points here are a plurality of points that can acquire three-dimensional data indicating the shape of the processed member by using these points (or can reproduce the three-dimensional data indicating the shape of the member). Preferably there is. However, one or more points may be other points. The three-dimensional data indicating the shape may be considered as a so-called three-dimensional model. The information indicating the point on the surface of the processed member is, for example, coordinates indicating the point. This coordinate is a three-dimensional coordinate. The point information may further include identification information for identifying the point.

  The one or more pieces of analysis information are information obtained by performing a predetermined analysis on a member processed using a mold, and one or more pieces of point information on the surface of the processed member are included. It is the information of the analysis result obtained about each point shown. The analysis designated in advance is, for example, stress analysis, plate thickness analysis, or strain analysis. The analysis designated in advance may be a plurality of different analyses. The analysis information for each point indicated by the point information is, for example, tertiary indicating the shape of the virtual workpiece using three-dimensional data indicating the shape of the mold and one or more member data that is information on the member to be processed. It can be acquired by acquiring information such as original data and performing analysis processing such as finite element analysis on the virtual workpiece. In addition, here, calculating the thickness information about one or more points of the processed product using the three-dimensional data indicating the shape of the virtual processed product is also considered as the analysis processing. The member data is, for example, information such as the material characteristics of the member and the thickness of the member before processing. For example, the stress analysis information can be obtained by performing press working simulation. The analysis information about the distortion and the like can be acquired in the same manner. The three-dimensional data indicating the shape of the member processed by the mold is, for example, a simulation using the three-dimensional data indicating the shape of the mold, or the three-dimensional data and one or more member data of the processed member. Can be obtained by performing When the three-dimensional data indicating the shape of the member processed by the mold is the three-dimensional data represented by the mesh as described above, and the point information is information indicating the node of the mesh data, each point The analysis information corresponding to the information is, for example, analysis information about one or more nodes of mesh data indicating the shape of a member processed with a mold. In addition, when the three-dimensional data etc. which show the shape of the virtual workpiece processed with the metal mold | die are prepared beforehand, you may use for acquiring the analysis result similar to the above using this three-dimensional data. . In addition, it is well-known about the analysis process, simulation, etc. which acquire the analysis information about the stress about one or more points of an object surface, plate | board thickness, and distortion using the three-dimensional data which show the shape of the processed member. Therefore, the description is omitted here.

  Note that the point stress information may further include three-dimensional data indicating the shape of the member processed by the mold. That is, it may further have three-dimensional data of a virtual processed product.

  The point analysis information storage unit 110 is preferably a nonvolatile recording medium, but can also be realized by a volatile recording medium. The same applies to the other storage units described below.

  In this embodiment, as an example, a case where point analysis information acquired by a point analysis information acquisition unit 111 described later is accumulated in the point analysis information storage unit 110 will be described. The process in which point analysis information is accumulated does not matter. For example, it goes without saying that point analysis information acquired by another analysis device or the like (not shown) may be stored in advance.

  The point analysis information acquisition unit 111 acquires point analysis information for a shape obtained by processing a member with one mold. One mold may be considered as a pair of molds constituted by a concave portion and a convex portion having a shape corresponding to the shape of the concave portion. The point analysis information acquisition unit 111 may acquire the point analysis information in any way. Hereinafter, as an example of the point analysis information acquisition process, points are obtained by performing analysis using the three-dimensional data of the mold and data such as material characteristics (for example, data of material characteristics) regarding the member to be processed. A case where analysis information is acquired will be described.

  For example, the point analysis information acquisition unit 111 first stores the three-dimensional data (for example, mesh data) of one mold that is stored in advance in a storage medium (not shown) and the storage medium (not shown) in advance. When one member is processed with the above-mentioned one mold using information on the thickness of one member to be processed, information on the member material such as the material of the member and its characteristic value, etc. Three-dimensional data (for example, mesh data) indicating the shape of the obtained member is acquired. That is, three-dimensional data of a virtual processed product obtained by processing is acquired. In addition, when acquiring the three-dimensional data, the information about the member such as the information indicating the plate thickness of the above-described one member to be processed that is accumulated in advance in a storage medium or the like that is not illustrated in advance is further provided if necessary. It may be used. In addition, when three-dimensional data indicating the shape of a workpiece obtained using a single mold is prepared in advance, the three-dimensional data of the workpiece may be read from a storage medium (not shown).

  Then, processing for analyzing the stress, plate thickness, distortion, etc. by the finite element analysis as described above using the acquired three-dimensional data of the processed product and the information on the one member to be processed as described above. And obtain one or more sets of analysis information on one or more points on the surface of the processed member and information (for example, coordinates) indicating each of the one or more points. In the above analysis, mesh data is often used as the 3D data to be analyzed. Therefore, if the 3D data is not mesh data, it is converted to mesh data as necessary. You can do it. The one or more points are, for example, nodes of three-dimensional data (for example, mesh) indicating the shape of the workpiece, and the analysis information about each point is, for example, the analysis result of each node obtained by the above analysis. Value. Information on the member (that is, member data) is information indicating the thickness of the member before processing, information on the material of the member, material properties of the member (Young's modulus, Poisson's ratio, mass density), and the like. For known techniques for analyzing stress and the like, refer to the following documents. ("Chapter 8 Analysis Exercises by Structural Analysis System", [online], [Search February 10, 2012], Internet <URL: http://www.tetras.uitec.ehdo.go.jp/download/ kankoubutu / i-109-09.pdf>)

  Then, the point analysis information acquisition unit 111 has one or more pairs of point information and analysis information, and mold analysis information of a mold to be analyzed and member identification information of a member in association with each other. Information is accumulated in the point analysis information storage unit 110.

  For example, the point analysis information acquisition unit 111 acquires point analysis information regarding a mold and a member corresponding to the mold identification information and the member identification information specified by the user. For example, the point analysis information acquisition unit 111 associates the mold three-dimensional data (for example, mesh data) and the member identification information stored in a storage medium (not shown) in association with the mold identification information. From the information about the member stored in the table, the mold identification information specified by the user and the three-dimensional data of the mold corresponding to the member identification information and the information about the member are read out, respectively, Analyze the analysis. The designation of the mold identification information and the member identification information used for analysis by the user is received by, for example, a receiving unit (not shown).

  The server reception unit 112 receives a set of mold identification information and member identification information transmitted from the terminal device 12. When one piece of mold identification information is associated with identification information of a processed product photographed by the terminal device 12 to be described later, the identification information of the processed product is received instead of the mold identification information. Anyway. The server receiving unit 112 is usually realized by a wireless or wired communication means, but may be realized by a means for receiving a broadcast. The server reception unit 112 may or may not include a communication device.

  The acquisition unit 113 performs point analysis on one or more sets of one or more pieces of point information corresponding to the mold identification information and member identification information received by the server reception unit 112 and analysis information corresponding to each piece of point information. Obtained from the information storage unit 110. The acquisition unit 113 can usually be realized by an MPU, a memory, or the like. The processing procedure of the acquisition unit 113 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The server transmission unit 114 transmits one or more sets of point information and analysis information acquired by the acquisition unit 113 to the terminal device 12. Specifically, it transmits to the terminal device 12 which becomes a transmission source of the set of mold identification information and member identification information. The server transmission unit 114 is usually realized by a wireless or wired communication unit, but may be realized by a broadcasting unit. The server transmission unit 114 may or may not include a communication device.

  The terminal device 12 is, for example, a mobile phone, a portable information terminal, a so-called smartphone, or a computer. The terminal device 12 is preferably a portable terminal device.

  The imaging unit 121 acquires one or more captured images that are images obtained by capturing a processed product obtained by processing a member using a mold. The processed product here is an actual processed product. The processed product is a concept including a molded product. The imaging unit 121 acquires, for example, at least a plurality of captured images in which one of the captured positions or directions is different for one processed product. The plurality of photographed images are images photographed from different positions around the processed product, for example. The captured image is usually a still image, but may be a moving image. The imaging unit 121 may be considered as, for example, an image sensor or a camera provided in the terminal device 12. The file format and resolution of the captured image are not limited. A camera such as a digital camera connected to the terminal device 12 via a signal line such as USB or a wireless or wired network may be considered as the imaging unit 121 of the terminal device 12. Alternatively, an input interface connected to these cameras may be considered as the imaging unit 121. The imaging unit 121 accumulates the acquired captured image in the captured image storage unit 122 as an example here.

  The captured image storage unit 122 stores one or more captured images acquired by the imaging unit 121. Here, storing is a concept including temporary storage.

  The receiving unit 123 receives mold identification information of a mold used for processing the processed product photographed by the imaging unit 121 and member identification information of a member used for processing. For example, the reception unit 123 receives mold identification information and member identification information input by a user or the like via an input device (not shown) such as a keyboard or a touch panel. The accepting unit 123 may accept the mold identification information that is read and output from the barcode indicating the mold identification information prepared in association with the mold or the like by the barcode reader. Similarly, the receiving unit 123 may receive member identification information that is read and output from a barcode indicating member identification information prepared in association with a member or the like by a barcode reader. Note that the receiving unit 123 may receive mold identification information or member identification information read by the photographing unit 12 or another processing unit (not shown) from the barcode image photographed by the photographing unit 121 as described above.

  Note that the receiving unit 123 may also receive information other than mold identification information and member identification information. For example, the receiving unit 123 may receive an input or the like for instructing output of information related to the analysis result of the processed product from the user or the like.

  The input means may be anything such as a numeric keypad, a keyboard, a mouse, a menu screen, or a barcode reader. The receiving unit 123 can be realized by a device driver of an input unit such as a numeric keypad, a keyboard, or a barcode reader, or control software for a menu screen.

  The terminal transmission unit 124 transmits the mold identification information and the member identification information received by the reception unit 123 to the server device 11. The terminal transmission unit 124 is usually realized by a wireless or wired communication unit, but may be realized by a broadcasting unit. The terminal transmission unit 124 may or may not include a communication device.

  The terminal receiving unit 125 receives one or more sets of point information and analysis information from the server device 11. Specifically, the server apparatus 11 receives one or more sets of point information and analysis information corresponding to the set of mold identification information and member identification information transmitted by the terminal transmission unit 124. The terminal receiving unit 125 is usually realized by a wireless or wired communication means, but may be realized by a means for receiving a broadcast. The terminal receiving unit 125 may or may not include a communication device.

  The imaging processed product shape data acquisition unit 126 is three-dimensional data indicating the shape (here, a three-dimensional shape) of one processed product that is an imaging target, using one or more captured images acquired by the imaging unit 121. A certain imaging processed product shape data is acquired. The imaging processed product shape data is three-dimensional data composed of meshes, curves, and the like as described above. For example, the imaging processed product shape data is acquired using one or more captured images of one processed product acquired by the imaging unit 121 and accumulated in the captured image storage unit 122.

  The imaging processed product shape data acquisition unit 126 normally acquires imaging processed product shape data using a plurality of captured images of one processed product. However, when the approximate shape of the processed product is known in advance (for example, when it is known that the processed product has a rotational shape, the imaged processed product shape data is acquired from one captured image. Since a technique for acquiring three-dimensional data indicating a three-dimensional shape of a subject from one or more captured images is a known technique, a detailed description thereof is omitted here. Please refer to the following documents, etc., for the technology to acquire the three-dimensional data indicating the three-dimensional shape of the subject from the image (Non-patent documents: "Autodesk 123D-123D Catch turn photos into 3D models", [ online], [February 10, 2012 search], Internet <URL: http://www.123dapp.com/catch>), (Non-patent literature: "3D photographic application measurement system: Kurabo Industries / Kuraves-MD (Claves-MD) ", [online], Search on February 10, 2012], Internet <URL: http://www.kurabo.co.jp/el/3d/kuraves_md_01.html>) (Non-patent literature: "Motion Portrait Inc .: Motion Port "Rate", [online], [Search February 10, 2012], Internet <URL: http://www.motionportrait.com/about/>). If the captured image is a moving image, The imaged processed product shape data may be acquired using one or more frame images in the moving image.

  In addition, every time the imaging unit 121 acquires one captured image, the imaging processed product shape data acquisition unit 126 uses this captured image or another one already acquired for the same processed product as this captured image. The imaged processed product shape data may be acquired using the above captured image. In this case, the imaging processed product shape data acquired immediately before may be deleted or overwritten, for example. By doing in this way, every time an image is taken, imaged processed product shape data is acquired in real time.

  Note that the minimum number of images required when acquiring the imaged processed product shape data from the captured image differs depending on the algorithm used, and therefore the number of captured images acquired by the capturing unit 121 reaches this minimum number of images. It is preferable that the imaging processed product shape data acquisition unit 126 acquires the imaging processed product shape data only when the image processing product shape data is acquired.

  The projection setting information acquisition unit 127 acquires projection setting information for the captured image. The projection setting information is information indicating a setting for obtaining a projection drawing obtained by projecting the shape indicated by the imaging processed product shape data acquired by the imaging processed product shape data acquisition unit 126, and the imaging processed product shape in the projection drawing. This is information indicating a setting for obtaining a projection in which the shape indicated by the data is the same as the shape of the processed product in the captured image. For example, the projection setting information is information used to generate a two-dimensional image obtained by projecting a three-dimensional shape indicated by three-dimensional data such as an imaging processed product shape data, and indicates information such as a projection method and a plane to be projected. It is. The projection diagram is, for example, a two-dimensional image obtained by rendering image processing product shape data that is three-dimensional data. Note that the same shape here means that the processed product in the photographed image and the processed product in the projected view include not only the shape but also the size, position in each image, etc. Means the same.

  For example, the projection setting information acquisition unit 127 acquires projection setting information for one or more captured images of the captured images used when acquiring the imaging processed product shape data. For example, the projection setting information acquisition unit 127 includes image processing product shape data in the projection diagram for each of one or more of the captured images used when the image processing product shape data acquisition unit 126 acquires the image processing product shape data. Projection setting information is acquired in which the shape of the processed product to be shown is the same as the shape of the processed product in the captured image.

  The projection setting information acquired for one of the captured images obtained by capturing the processed product processed using the mold is, for example, the normal vector of the image capturing unit 121 (for example, camera) used for capturing. The information includes a combination with the rotation angle of the imaging unit 121. The normal vector is, for example, a normal vector of the light receiving surface (light receiving element) of the imaging unit 121. The normal vector may be considered as the direction in which the optical axis of the imaging unit 121 extends. Further, the rotation angle is a rotation angle of the imaging unit 121 with respect to the optical axis of the imaging unit 121. In this case, the orientation and angle of view indicated by the projection setting information are obtained from the position of the imaging unit indicated by the projection setting information in the virtual three-dimensional space in which the imaging artifact shape data is arranged. By performing rendering by setting, it is possible to obtain a projection view in which the shape of the portion corresponding to the imaged processed product shape data is the same as the shape of the processed product in the captured image. The projection setting information may further include information on the angle of view of the imaging unit 121, information on the distance between the imaging unit 121 and the processed product, and the like. Note that information on the angle of view of an optical system such as a lens used by the imaging unit 121 may be acquired as the information on the angle of view. In addition, information on the distance between the imaging unit 121 and the processed product may be input by a user or the like via a reception unit (not shown). This distance may be fixed. In the case of fixing, the user may image the actual processed product using the imaging unit 121 from a position approximately the same distance as this distance.

Also, the projection setting information specifies the position, orientation (for example, normal vector), and width (for example, values indicating height and width) of the surface on which the imaged work product shape data is projected. It may be information.
Note that the projection setting information includes size information indicating the size of the captured image, part designation information indicating each part (for example, a point such as a node) of the imaged processed product shape data of the processed product that has been captured. It may be a combination with partial coordinates, which are coordinates indicating the position where each part of the imaging processed product shape data designated by the part designation information in the photographed image. The part designation information is the coordinates (three-dimensional coordinates) of each part of the imaging processed product shape data. Using such projection setting information, for example, each part of the imaging processed product shape data indicated by the part designation information is in the coordinates indicated by each partial coordinate in the image having the size indicated by the size information of the projection setting information. If the projection drawing in which the image processing product shape data is arranged is generated by rendering so that each part of the image processing product shape data indicated by the part designation information corresponding to each part coordinate is arranged, the image processing product shape It is possible to obtain a projection view in which the shape of the part corresponding to the data is the same as the shape of the processed product in one captured image.
Imaging processed product shape data Imaging processed product shape data Imaging processed product shape data Imaging processed product shape data

  Since the imaged processed product shape data is generated using a plurality of images, it is not completely the same as the actual processed product, so the shape of the processed product in one captured image and the one captured image Even if there is a difference in the degree of error in the shape of the processed product in the projection drawing (that is, the rendered drawing) acquired for the imaging processed product shape data using the acquired projection setting information, the same shape is used here. You may think that there is.

  The projection setting information acquisition unit 127 may acquire the projection setting information in any way. For example, if possible, a three-dimensional sensor such as a three-axis gyro sensor that detects the normal direction of the imaging unit 121 and the rotation direction is provided for the imaging unit 121. The vector indicating the normal direction of the image capturing unit 121 acquired by, the value indicating the rotation direction, and the like may be acquired as projection setting information. Alternatively, such a three-dimensional sensor may be provided in the terminal device 12.

  Further, when the imaging processed product shape data acquisition unit 126 performs processing for acquiring imaging processed product shape data using a plurality of captured images, each captured image is used as a projection view of the imaging processed product shape data to be acquired. In order to handle and acquire imaging processed product shape data, at least temporarily, the imaging processed product shape data is associated with each captured image, so imaging to obtain the same projected image of the processed product shape data as each captured image Values such as the normal vector of the unit 121 and the rotation angle of the imaging unit 121 can be acquired. For this reason, the projection setting information acquisition unit 127 receives the above-described projection setting information (for example, the normal vector of the imaging unit 121 and information on the rotation angle) during the process of acquiring the imaging processed product shape data. It can be acquired. The process for acquiring the projection setting information in this way is a known technique.

  Further, when the imaging processed product shape data acquisition unit 126 performs processing for acquiring imaging processed product shape data using a plurality of captured images, each captured image is used as a projection view of the imaging processed product shape data to be acquired. In order to handle and acquire imaging processed product shape data, the imaging processed product shape data is associated with each captured image at least temporarily. For this reason, the projection setting information acquisition unit 127 performs the above-described projection setting information (for example, information on the size of the captured image and the shape of the imaged processed product in the captured image) during the process of acquiring the imaged processed product shape data. It is possible to obtain the coordinates where each part of the data is located). The process for acquiring the projection setting information in this way is a known technique. Further, the projection setting information acquisition unit 127 may acquire (accept) such projection setting information from the imaging processed product shape data acquisition unit 126. Alternatively, the projection setting information acquisition unit 127 acquires the same processing as the processing performed when the imaging processed product shape data acquisition unit 126 acquires imaging processed product shape data from one or more captured images. It is also possible to acquire the projection setting information for the photographic image that is the acquisition target by performing it for one or more photographic images including the photographic image.

  When the projection setting information acquisition unit 127 acquires the projection setting information acquired during the process in which the imaging processed product shape data acquisition unit 126 acquires the imaging processed product shape data, for example, the imaging processed product shape data The acquisition unit 126 and the projection setting information acquisition unit 127 may be realized by a single processing unit or the like. In the projection setting information acquired by the projection setting information acquisition unit 127, the projection setting information acquired for one captured image is stored in a storage medium (not shown) in association with the one captured image.

  Each time the imaging unit 121 acquires one captured image, the projection setting information acquisition unit 127 obtains projection setting information for acquiring a projection map corresponding to the one captured image for the one captured image. You may make it acquire.

  The analysis processed product shape data acquisition unit 128 acquires analysis processed product shape data, which is three-dimensional data indicating the shape corresponding to the mold indicated by the mold identification information received by the receiving unit 123. Specifically, the mold shape acquisition unit 128 uses the point information received by the terminal reception unit 125 to acquire analysis processed product shape data that is three-dimensional data indicating the shape corresponding to the mold. The three-dimensional data indicating the shape is information such as a mesh or a curve indicating the shape as described above.

  The shape corresponding to the mold is, for example, the shape of a processed product obtained by processing using the mold indicated by the mold identification information received by the receiving unit 123. For example, the shape corresponding to the mold is the shape of the processed product obtained when the member indicated by the member identification information received by the receiving unit 123 is processed using the mold indicated by the mold identification information. The three-dimensional data indicating the shape corresponding to the mold is, for example, three-dimensional data indicating the shape of a workpiece created by simulation or the like using the three-dimensional data of the mold, or the corresponding three-dimensional data. It is data. For example, the three-dimensional data is three-dimensional data indicating the shape of the virtual workpiece that is the target of the analysis specified in advance. Since each piece of point information received by the terminal receiving unit 125 is information indicating a point (for example, a node) on the surface of the member when the member is processed using a mold as described above, each piece of point information is used. Thus, three-dimensional data indicating the shape corresponding to the mold can be acquired. For example, when the point information is node information of three-dimensional data indicating a design drawing of a processed product, the shape corresponding to the mold acquired by the mold shape acquisition unit 128 is a three-dimensional display indicating the design drawing of the processed product. It becomes data. For example, the mold shape acquisition unit 128 acquires a three-dimensional mesh by connecting the coordinates indicated by the point information in accordance with a rule specified in advance. Since a technique for acquiring data of a three-dimensional shape in which these three-dimensional coordinates are located on the surface from a plurality of three-dimensional coordinates is a known technique, a detailed description of the processing is omitted here. Note that the point information used when acquiring the analytical processed product shape data indicates the surface point of the analytical processed product shape data acquired by the analytical processed product shape data acquisition unit 128.

  The analysis processed product shape data acquisition unit 128 does not need to acquire the analysis processed product shape data using only the point information received by the terminal reception unit 125. For example, the terminal transmission unit 124 transmits the data from the server device 11. The server apparatus 11 may receive three-dimensional data indicating the three-dimensional shape of the workpiece molded by the mold corresponding to the mold identification information. In this case, the server device 11 stores information in which the three-dimensional data of the processed product and the mold identification information are associated with each other in advance in a storage unit such as the point analysis information storage unit 110, and the server reception unit When 112 receives the mold identification information, it is necessary to transmit the three-dimensional data of the processed product corresponding to the mold identification information to the terminal device 12.

The conversion unit 129 performs coordinate conversion of the analysis processed product shape data so that the analysis processed product shape data overlaps the imaging processed product shape data. Since the analysis processed product shape data and the imaging processed product shape data are three-dimensional data obtained with respect to the same mold, they can be superimposed. However, the analysis processed product shape data created using the point information and the imaging processed product shape data are usually different in position, orientation, size, etc. in the virtual three-dimensional space. For this reason, the conversion unit 129 arranges the analysis processed product shape data and the imaging processed product shape data in the same three-dimensional coordinate system, and appropriately determines the position, size, and orientation of the shape indicated by the analysis processed product shape data. The analysis processed product shape data is overlaid with the imaging processed product shape data. Then, information (for example, information on three-dimensional coordinates) indicating the position of each part (for example, a node such as a mesh or a curve) of the analysis processed product shape data in the overlapped state is acquired. In this way, one or more pieces of information for designating the position of each part of the new analytical processed product shape data obtained by moving the analytical processed product shape data so as to overlap with the imaging processed product shape data is obtained by coordinate conversion. It is the obtained analytically processed product shape data. The conversion unit 129 performs coordinate conversion of one or more point information indicating one or more points on the surface of the shape indicated by the analysis processed product shape data before coordinate conversion together with the coordinate conversion of the analysis processed product shape data, for example, The analysis information associated with the point information before the coordinate conversion is associated with this point information after the coordinate conversion, and the analysis information associated with the shape indicated by the analysis processed product shape data before and after the coordinate change. The relative position with respect to the analysis processed product shape data of the point associated with is not changed. In addition, when the point indicated by the point information is the node information of the analysis processed product shape data, the point identification information indicating the point information is associated with each contact of the analysis processed product shape data. Thus, even if the coordinates of each node in the analysis processed product shape data are coordinate-converted, the correspondence between each node and the point identification information is maintained, and this indicates the point information in the analysis processed product shape data after coordinate conversion. It becomes possible to maintain the correspondence between the points (that is, each node here) and the analysis information. For example, the analysis information associated with one node of the analysis processed product shape data before coordinate conversion is the analysis information associated with the node corresponding to this one node of the analysis processed product shape data after coordinate conversion. Information.
The process of superimposing the analysis processed product shape data on the imaging processed product shape data is performed by, for example, extracting feature portions (for example, three feature points, the position of the center of gravity, the longitudinal direction, etc.) of each shape data. Match the coordinate system to match.

  Hereinafter, an example of processing for superimposing the analysis processed product shape data on the imaging processed product shape data will be described. First, for the three-dimensional data indicating each shape, the longest length direction (referred to as the first direction) is detected, and the size of the analysis processed product shape data is set so that the length in that direction is the same length. Is enlarged (enlarged so that all of the three-dimensional directions are enlarged at the same magnification). Then, the positions of the enlarged analysis processed product shape data in the virtual three-dimensional space are arranged so that both ends of the enlarged analysis processed product shape data in the first direction overlap with both ends of the imaging processed product shape data in the first direction. Move. The movement of the position here may include rotational movement. Further, a direction orthogonal to the first direction and having the longest length (hereinafter referred to as a second direction) is detected for the three-dimensional data indicating the respective shapes, It is determined whether or not the intersection between the first direction and the second direction matches the intersection between the first direction and the second direction of the imaging processed product shape data. The analysis processed product shape data is rotated 180 degrees around the midpoint of the first direction of the data. If they match, do not rotate. Next, in the virtual three-dimensional space, the analysis work product shape data is set with the first direction as the rotation axis so that the second direction of the analysis work product shape data overlaps the second direction of the imaging work product shape data. Rotate. The analytically processed product shape data thus obtained is the coordinated transformed analytically processed product shape data.

  Note that the process for superimposing the analysis processed product shape data on the imaging processed product shape data may be performed using a matching process of shape data indicating a three-dimensional shape. Since the matching process of the shape data of the three-dimensional shape is a known technique, a detailed description thereof is omitted here. Please refer to the following technology for matching processing of 3D shape data. (Non-patent literature: Kazuyoshi Kodaka, 2 others, "Feature extraction and matching of 3D wireframe model", [online], [Search February 23, 2012], Internet <URL: http: // www. shalab.phys.waseda.ac.jp/pub/personal/odaka/odaka1.pdf>, (Non-Patent Document 2: Ritsumasa Sagawa, 4 others, “Real time by incremental mesh modeling and its hierarchical recognition method” Research on 3D object recognition system ", [online], [Search on February 13, 2012], Internet <URL: http://www.am.sanken.osaka-u.ac.jp/~sagawa/pdf /jrsj02.pdf>).

  In the process of converting the coordinates so as to overlap, when determining whether or not the positions, distances, and the like of the coordinates and the like of the analysis processed product shape data and the imaging processed product shape data match, It may be determined that even if there is a difference in error level, a match is made by taking into account a predetermined error or the like. Alternatively, even if there is a difference in error level, it may be determined that they match. For example, when determining a match or the like, a coordinate value obtained from the shape three-dimensional data or the like or a value less than a predetermined number of distances may be ignored or rounded off.

  It should be noted that the coordinate conversion of the imaging processed product shape data is performed so that the imaging processed product shape data overlaps with the analytical processed product shape data, and the analytical processed product shape data is converted so that the analytical processed product shape data overlaps with the imaging processed product shape data. You may make it think that it is the same thing as coordinate transformation. However, in this case, the projection setting information is also subjected to coordinate conversion in accordance with the coordinate conversion of the imaging processed product shape data so that the relative positional relationship with the imaging processed product shape data, the relationship of the size ratio, and the like do not change. This may also be considered to be included in the process of coordinate conversion of the imaging processed product shape data. In the case of pattern matching or the like, a case where the degree of matching is the highest may be determined as overlapping.

  The analysis image generation unit 130 analyzes the analysis processed product shape data coordinate-converted by the conversion unit 129, analysis information corresponding to point information indicating one or more points on the surface of the three-dimensional shape indicated by the analysis processed product shape data, and projection Using one of the setting information, an analysis image, which is an image of a projection diagram showing information on the analysis result indicated by the analysis information on the surface of the shape indicated by the analysis processed product shape data, according to the setting indicated by the projection setting information Generate. Coordinate-transformed analysis processed product shape data is ideally the same solid shape with the same size, arranged in the same position in the same virtual 3D coordinate system as the imaged processed product shape data, in the same direction Therefore, the projection drawing of the analysis processed product shape data is generated using one of the projection setting information acquired for the imaging processed product shape data. However, normally, the imaging processed product shape data and the analysis processed product shape data are data indicating the shape of the same processed product, but the imaging processed product shape data is an image obtained by photographing the actually created processed product. It is created from the above, and does not completely match the analysis processed product shape data. One piece of projection setting information used here corresponds to, for example, an arbitrary photographed image among one or more pieces of photographing information used when the image pickup processed product shape data acquiring unit 126 acquires the image pickup processed product shape data. Projection setting information or projection setting information corresponding to a captured image designated by the user via a reception unit (not shown). The generation of an analysis image, which is a projection view here, is so-called rendering using a setting indicated by one projection setting information using analysis processed product shape data. In the process of generating the analysis image, for example, the optical axis direction coincides with the direction indicated by the normal vector included in one projection setting information, and the rotation angle with respect to the optical axis is the rotation angle indicated by the projection setting information. A two-dimensional projection of the shape indicated by the analytical work shape data by projecting the area where the shape indicated by the analytical work shape data viewed from the virtual viewpoint (camera) set in this way onto a two-dimensional plane This is a process for generating an image. That is, it is a rendering process of analysis processed product shape data from a virtual viewpoint indicated by the projection setting information. When the projection setting information includes the angle of view of the viewpoint (camera) used for rendering and the distance between the viewpoint and the analysis processed product shape data, the analysis processed product shape data is rendered using these information. You may make it do. The process of rendering three-dimensional data using such projection setting information is a known technique. Moreover, the process which produces | generates an analysis image may be a process which produces | generates the two-dimensional image which projected the shape which analysis work product shape data shows on the projection surface which one projection setting information shows, for example.

  Here, when acquiring an image of a projection diagram for a three-dimensional shape indicated by normal three-dimensional data, an image is generated by mapping an image such as a texture image on the surface of the three-dimensional shape. Using the analysis information corresponding to the point information indicating one or more points on the surface of the three-dimensional shape indicated by the coordinate-converted analytical work product shape data, information on the analysis result indicated by the analysis information is obtained as the analysis work product shape data. The analysis image which is the image of the projection figure of the state arrange | positioned on the surface of the three-dimensional shape which shows is produced | generated.

  The information related to the analysis result may be a value of the analysis result indicated by the analysis information (for example, stress, plate thickness, strain) or the like (for example, the analysis result) It may be information indicating a distribution or an isoline of analysis result values). For example, the analysis image generation unit 130 performs analysis corresponding to each piece of point information at a position (for example, on each node) indicated by one or more pieces of point information on the surface of the three-dimensional shape indicated by the coordinate-converted analytical work product shape data. An analysis image in which the value indicated by the information is arranged is generated by rendering. Also, a mark indicating that the maximum or minimum value of the analysis information or the maximum or minimum value of the analysis information is at the position indicated by the point information indicating the maximum or minimum value of the analysis information. An analysis image in which images showing colors and colors are arranged may be generated by rendering. The position indicated by the point information may be considered as a position having a clear correspondence relationship with the point information (for example, a position connected to the position indicated by the point information by a line or the like). An analysis image indicating the distribution of analysis results may be generated. For example, the point information is divided into one or more groups for each corresponding analysis value range, and a range including one or more points indicated by the point information belonging to the same group is displayed differently from other point information (for example, May be displayed in different colors or surrounded by lines. Further, analysis information in which one or more lines connecting points having the same analysis result value indicated by the analysis information by a straight line or a curve, that is, an isoline of the analysis result value may be generated by rendering.

  A function of the coordinates on the straight line or curve connecting the two points and the value of the analysis information is calculated from the two pieces of point information of the analysis processed product shape data and the values of the respective analysis information. Is used to acquire analysis information about positions (points) other than the position indicated by the point information, and to acquire an analysis image in which information on analysis results including analysis information about such points is arranged. Also good. For example, the analysis information about the position other than the position indicated by the point information acquired in this way is arranged on the surface of the three-dimensional shape, or the acquired analysis information is used to acquire the isoline, You may make it produce | generate the analysis image which has arrange | positioned the line. This isoline may be a line having a different color or the like depending on the value. Moreover, you may make it acquire the average value of the value of the analysis information corresponding to two point information as analysis information about the intermediate position of the two point information of analysis processed product shape data. In addition, by using the average value for the new point acquired in this way and repeating the process of acquiring the analysis information about the intermediate point with other points, a point other than the position indicated by the original point information Analysis information can be acquired, and an analysis image may be generated using this analysis information.

  As described above, every time the imaging unit 121 acquires one captured image, the imaging processed product shape data acquisition unit 126 acquires imaging processed product shape data using at least this one captured image, and sets the projection. When the information acquisition unit 127 acquires the projection setting information for the one captured image, the analysis image generation unit 130 uses the projection setting information acquired by the projection setting information acquisition unit 127 for the one captured image. An analysis image may be generated.

  The process of displaying information such as numerical values or displaying isolines on the surface of the three-dimensional data indicating the three-dimensional shape is known as a process of three-dimensional graphics, etc. Detailed processing is omitted.

  Further, the analysis image generation unit 130 may further arrange information other than information related to the analysis information on the surface of the three-dimensional shape indicated by the analysis processed product shape data. For example, a predetermined texture may be arranged, or an analysis image further displaying a mesh of analysis processed product shape data may be acquired.

  The output unit 131 outputs an image obtained by superimposing the captured image and the analysis image corresponding to the projection setting information used when the analysis image generation unit 130 generates the analysis image. The projection setting information associated with one photographed image is a setting for obtaining a projection map having the same arrangement as that of the one photographed image when rendering the imaging processed product shape data and obtaining the projection map. Information. Here, since one projection setting information is used when acquiring an analysis image, which is a projection view, from the analysis processed product shape data whose coordinates are converted so as to overlap with the imaging processed product shape data, the analysis image generation unit 130 uses the one projection setting information. The generated analysis image is a projection view of analysis processed product shape data having the same arrangement as that of the one captured image corresponding to the one projection setting information. For this reason, when the captured image and the analysis image are overlapped, for example, information on the analysis result indicated by the analysis image (for example, an isoline of the value of the analysis result, etc.) on the region where the processed product of the captured image is displayed. ) Will be placed.

  When the output unit 131 generates an image obtained by superimposing the captured image and the analysis image, it does not matter how the images are combined as long as the information indicated by both images can be visually grasped. Absent. For example, when the information other than the part where the information related to the analysis result of the analysis image is arranged has transmission information, the analysis image may be simply superimposed on the captured image. Further, the captured image and the analysis image may be combined in a specific image mode such as a multiplication mode, a screen mode, or an overlay mode.

  In addition, when the projection setting information used when generating the analysis image is the projection setting information corresponding to the photographed image specified by the user or the latest photographed image acquired by the photographing unit 121, for example, by this user. The designated photographed image or the latest photographed image acquired by the photographing unit 121 becomes a photographed image corresponding to the projection setting information used when the analysis image generating unit 130 generates the analysis image.

  Each time the imaging unit 121 acquires one captured image, the output unit 131 outputs the one captured image and the analysis image acquired by the analysis image generation unit 130 using the one captured image. A superimposed image may be output.

  Output here means display on a display, projection using a projector, printing on a printer, transmission to an external device, storage on a recording medium, processing results to other processing devices or other programs, etc. It is a concept that includes delivery.

  The output unit 131 may or may not include an output device such as a display or a printer. The output unit 131 can be realized by output device driver software, or output device driver software and an output device.

  Next, operation | movement of the server apparatus 11 of the information processing system 1 is demonstrated using the flowchart of FIG.

  (Step S101) The point analysis information acquisition unit 111 determines whether an instruction to acquire point analysis information has been received via a reception unit (not shown) or the like. If accepted, the process proceeds to step S102. If not accepted, the process proceeds to step S107.

  (Step S102) The point analysis information acquisition unit 111 receives mold identification information, three-dimensional data indicating the three-dimensional shape of the mold indicated by the mold identification information, member identification information, and the like via a reception unit (not illustrated). Then, it is determined whether or not a set of member data that is information such as material characteristics of the member corresponding to the member identification information has been received. If accepted, the process proceeds to step S103, and if not accepted, the process returns to step S102.

  (Step S103) The point analysis information acquisition unit 111 performs a simulation or the like for performing an analysis designated in advance using the three-dimensional data of the mold received in Step S102 and the member data, and uses the member data by the mold. A set of point information indicating one or more points on the surface of the three-dimensional shape obtained by processing and the analysis information at the point indicated by each point information is acquired. For example, one or more sets of point information indicating each node and analysis information acquired by analyzing each node of three-dimensional data such as a mesh indicating the three-dimensional shape of a processed product obtained by processing with a mold are acquired. To do.

  (Step S104) The point analysis information acquisition unit 111 has one or more sets of the point information and analysis information acquired in step S103, the mold identification information received in step S102, and the member identification information. Are accumulated in the point analysis information storage unit 110. Then, the process returns to step S101.

  (Step S <b> 105) The server reception unit 112 determines whether or not the mold identification information and the member identification information are received from the one terminal device 12. If received, the process proceeds to step S106. If not received, the process returns to step S101.

  (Step S106) The acquisition unit 113 searches the point analysis information storage unit 110 for one or more sets of point information and analysis information corresponding to the set of mold identification information and member identification information received in step S105. Get by.

  (Step S107) The server-side transmission unit 114 is a terminal device that is a transmission source of the mold identification information and the member identification information received in Step S105 for one or more pairs of the point information and the analysis information acquired in Step S106. 12 to send. Then, the process returns to step S101.

  In the flowchart of FIG. 3, the process ends when the power is turned off or the process is terminated.

  Next, operation | movement of the terminal device 12 of the information processing system 1 is demonstrated using the flowchart of FIG.

  (Step S201) The receiving unit 123 determines whether or not mold identification information has been received. If accepted, the process proceeds to step S202. If not accepted, the process returns to step S201.

  (Step S202) The receiving unit 123 determines whether or not member identification information has been received. If accepted, the process proceeds to step S203. If not accepted, the process returns to step S202.

  (Step S203) The terminal transmission unit 124 transmits the set of the mold identification information received in Step S201 and the member identification information received in Step S202 to the server device 11.

  (Step S <b> 204) The terminal reception unit 125 determines whether or not one or more sets of point information and analysis information have been received from the server device 11. If received, the process proceeds to step S205. If not received, the process returns to step S204.

  (Step S205) The terminal receiving unit 125 accumulates the received set of point information and analysis information in a storage unit such as a storage medium (not shown).

  (Step S206) The imaging unit 121 uses the mold indicated by the mold identification information received in Step S201 to photograph a processed product obtained by processing the member indicated by the member identification information received in Step S202. It is determined whether a captured image has been acquired. If acquired, the process proceeds to step S207. If not acquired, the process returns to step S206.

  (Step S207) The imaging unit 121 accumulates the captured image acquired in step S206 in the captured image storage unit 122.

  (Step S208) The imaging processed product shape data acquisition unit 126 determines whether or not the number of captured images acquired by the imaging unit 121 is equal to or greater than a predetermined minimum number. For example, the imaging processed product shape data acquisition unit 126 determines whether or not the number of captured images stored in the captured image storage unit 122 is equal to or greater than the minimum number. If it is equal to or greater than the minimum number, the process proceeds to step S209, and if it is less than the minimum number, the process returns to step S206.

  (Step S209) The imaging processed product shape data acquisition unit 126 acquires imaging processed product shape data using one or more captured images stored in the captured image storage unit 122.

  (Step S210) The projection setting information acquisition unit 127 acquires the projection setting information corresponding to the latest captured image acquired by the imaging unit 121. In addition to the projection setting information corresponding to the latest photographed image, one or more of the photographed images for which the corresponding projection setting information is not acquired from the photographed images stored in the photographed image storage unit 122 (for example, Projection setting information may also be acquired for all. Then, the projection setting information acquisition unit 127 stores the projection setting information acquired for the captured image in step S210 in a storage medium such as the captured image storage unit 122 in association with the captured image.

  (Step S211) The analysis processed product shape data acquisition unit 128 determines whether or not the analysis processed product shape data subjected to coordinate conversion has been acquired. If already acquired, the process proceeds to step S212. If not acquired, the process proceeds to step S214.

  (Step S212) The analysis processed product shape data acquisition unit 128 acquires the analysis processed product shape data using the point information accumulated in step S205.

  (Step S213) The conversion unit 129 performs coordinate conversion so that the information on the coordinates constituting the analysis processed product shape data acquired in step S212 overlaps with the imaging processed product shape data acquired in step S209. Note that the point information may also be coordinate-transformed in accordance with the coordinate transformation of the analysis processed product shape data.

  (Step S214) The analysis image generation unit 130 performs analysis-converted product shape data obtained by coordinate conversion, analysis information corresponding to the point information accumulated in step S205, and projection setting information corresponding to the latest photographed image acquired in step S210. And an analysis image is generated.

  (Step S215) The output unit 131 reads the latest captured image accumulated in step S207, and outputs an image in which the read captured image is superimposed on the analysis image generated in step S214. If another image has already been output, the output is updated with a new image. Then, the process returns to step S206.

  In the flowchart of FIG. 3, the process ends when the power is turned off or the process is terminated.

  Note that once processing for outputting an analysis image superimposed on a captured image is performed in step S215, imaging processed product shape data has already been obtained. For this reason, once the process of step S215 is performed, the imaging processing product that acquires the imaging processing product shape data from the captured image in order for the projection setting information acquisition unit 127 to obtain the projection setting information for the new captured image. When it is not necessary to use the process of the shape data acquisition unit 126, the process of step S209 may be omitted.

  Hereinafter, a specific operation of the information processing system 1 in the present embodiment will be described.

  FIG. 4 is a conceptual diagram of the information processing system 1. Here, it is assumed that the server device 11 is a computer and the terminal device 12 is a high-function mobile phone called a smartphone. Here, it is assumed that the terminal device 12 includes a monitor 120. For example, it is assumed that the server device 11 and the terminal device 12 are connected via a network such as the Internet.

  First, the user performs a predesignated operation for starting acquisition of point analysis information with respect to the server device 11, and further, is three-dimensional data indicating the shape of the mold with respect to the server device 11. It is assumed that CAD data, mold identification information of the mold, member data (material characteristics, data such as thickness) of the member processed by the mold, and member identification information of the member are input. .

  The point analysis information acquisition unit 111 performs a simulation or the like specified in advance using a mold indicated by the input three-dimensional data, and processes the data indicated by the input member data (that is, a three-dimensional shape (that is, Three-dimensional data indicating the three-dimensional shape of the processed product) is acquired. Here, for example, three-dimensional data composed of meshes is acquired. Further, a simulation for performing a predesignated analysis is performed using the three-dimensional data indicating the processed shape and the member data of the member to be processed, and the processed three-dimensional data Get analysis information about each node of the mesh.

  The point analysis information acquisition unit 111 acquires a plurality of sets of point information indicating the coordinates of each node from which the analysis information is acquired and point identification information that is identification information of each node, and the acquired point information and point identification information A plurality of point management information having each of the plurality of sets, the mold identification information received above, and the member identification information is accumulated in the point analysis information storage unit 110. That is, the point identification information in this specific example may be considered as node identification information. The point analysis information acquisition unit 111 also includes point identification information that is identification information of each node, analysis information acquired for each node indicated by each point identification information, mold identification information received above, and member identification A plurality of point analysis management information having information is accumulated in the point analysis information storage unit 110. In this specific example, the point analysis information is configured by associating the point management information and the point analysis management information with the point identification information.

  FIG. 5 is a diagram showing point management information stored in the point analysis information storage unit 110. The point management information includes items of “mold ID”, “member ID”, “point ID”, and “point information”. “Mold ID” is mold identification information. “Member ID” is member identification information. “Point ID” is point identification information, and here is node identification information of three-dimensional data indicating the shape of the processed member. “Point information” is point information, and here it is assumed to be the three-dimensional coordinates of the node indicated by “Point ID”. Note that xn, yn, and zn (where n is a natural number of 1 or more) in the “point ID” are an x-coordinate value, a y-coordinate value, and a z-coordinate value, respectively. The same applies to other coordinates described below.

  FIG. 6 is a diagram showing point analysis management information stored in the point analysis information storage unit 110. The point analysis management information includes items of “mold ID”, “member ID”, “point ID”, and “analysis information”. The “analysis information” further includes items of “stress”, “plate thickness”, and “strain”. The “mold ID”, “member ID”, and “point ID” are the same as those in FIG. “Analysis information” is analysis information about a virtual workpiece that is considered to be obtained when the member indicated by “member ID” is processed using the die indicated by “die ID”. Assume that the analysis of stress, plate thickness, and strain is performed as a predesignated analysis. “Stress” is information on the stress obtained by analyzing the stress at the point (here, the node) indicated by the corresponding “point ID”. The “stress” further includes items “σxx”, “σyy”, “σzz”, “τxy”, “τyz”, and “τzx” indicating values of components of the stress analysis result. It should be noted that xxn, yyn, zzn, xyn, yzn, and zxn (where n is a natural number of 1 or more) of “stress” are numerical values of analysis results. The “plate thickness” is information on the plate thickness obtained by analyzing the plate thickness at the point indicated by the corresponding “point ID”. “Distortion” is distortion information obtained by analyzing the distortion at the point indicated by the corresponding “point ID”. Note that tn and εn (where n is a natural number of 1 or more) are values of thickness and strain, respectively. Here, the strain value εn may be a vertical strain value, a shear strain value, or may include both values. The strain value εn may be considered to include a tensor component or the like.

  Next, the analysis result of the processed product obtained by the processing at the place (for example, factory, laboratory, etc.) where the user performs press forming processing on the metal member by using the mold. In order to investigate, it is assumed that an instruction to output the analysis result is given to the terminal device 12.

  FIG. 7 is a diagram (FIG. 7A) showing a press molding machine and a diagram (FIG. 7B) showing members to be press-molded. In the figure, it is assumed that a two-dimensional barcode 72 indicating mold identification information of a mold 71 used for processing is attached to the press molding machine 70. Further, it is assumed that a two-dimensional barcode 74 indicating member identification information of the member 73 is attached to the member 73.

  When the reception unit 123 or the like of the terminal device 12 receives an instruction to output the analysis result via an input device such as a touch panel (not shown), the output unit 131 causes the monitor 120 or the like to have a mold die used for processing. Displays a dialog or the like prompting for input of type identification information.

  FIG. 8 is a perspective view of the terminal device 12 as seen from the surface opposite to the surface on which the monitor 120 is provided. Here, for example, it is assumed that the terminal device 12 includes an imaging unit 121 having a lens (not shown) on the surface opposite to the surface on which the monitor 120 is provided. The imaging unit 121 is assumed to be a camera in this specific example. The imaging unit 121 also has a function as a two-dimensional barcode reader.

  When the user captures a two-dimensional barcode indicating the mold identification information by the imaging unit 121 in accordance with a dialog prompting the user to input the mold identification information of the mold, the imaging unit 121 reads and decodes the two-dimensional barcode. And mold identification information “P01” is acquired. The receiving unit 123 receives the mold identification information “P01” acquired by the imaging unit 121.

  Since the mold identification information has been acquired, the output unit 131 displays a dialog or the like for prompting input of the member identification information of the member to be processed on the monitor 120 or the like.

  It is assumed that the imaging unit 121 acquires the member identification information “M001” when the imaging unit 121 captures a two-dimensional barcode indicating the member identification information of the member 73 according to this dialog. The receiving unit 123 receives the mold identification information “P01” acquired by the imaging unit 121.

  Since the reception unit 123 has received the mold identification information “P01” and the member identification information “M001”, the terminal transmission unit 124 sets the combination of the received mold identification information “P01” and the member identification information “M001”. Is transmitted to the server apparatus 11 via the network.

  When the server reception unit 112 of the server device 11 receives the set of the mold identification information “P01” and the member identification information “M001” transmitted from the terminal device 12, the acquisition unit 113 performs the point analysis illustrated in FIG. Among the point management information stored in the information storage unit 110, the value of “mold ID” matches the received mold identification information “P01”, and the value of “member ID” is the received member. Point management information that matches the identification information “M001” is searched, and a set of “point ID” and “point information” is acquired from each of the detected one or more point management information. Here, as an example, a set of “point ID” and “point information” is acquired from all detected point management information. Further, the acquisition unit 113 sets the value of “mold ID” as the received mold identification information “P01” from the point analysis management information stored in the point analysis information storage unit 110 illustrated in FIG. Point analysis management information that matches and the value of the “member ID” matches the received member identification information “M001” is searched, and “point ID” and “analysis information” are detected from each of the detected point analysis management information. Get a pair with. Here, as an example, a set of “point ID” and “analysis information” is acquired from all detected point analysis management information.

  Furthermore, here, as an example, correspondence is made from a set of “point ID” and “point information” acquired from the point management information, and a set of “point ID” and “analysis information” acquired from the point analysis management information. “Point information” and “Analysis information” having the same “Point ID” are sequentially extracted, and one or more pieces of information having the extracted “Point information”, “Analysis information”, and “Point ID” corresponding thereto are obtained. Obtain sequentially. Here, this information is referred to as point analysis acquisition information. The point analysis acquisition information is information having point information and analysis information about the point indicated by the point information in association with each other. The server transmission unit 114 transmits the one or more point analysis acquisition information acquired by the acquisition unit 113 to the terminal device 12. However, it is not always necessary to acquire the point analysis acquisition information.

  The terminal receiving unit 125 receives the point analysis acquisition information transmitted from the server device 11 and accumulates it in a storage medium (not shown). The accumulation here may be temporary storage.

  FIG. 9 is a diagram illustrating the point analysis acquisition information received and accumulated by the terminal reception unit 125. The point stress acquisition information includes “point ID”, “point information”, and “analysis information”. The “analysis information” is the same as the “analysis information” in FIG.

  Since the terminal reception unit 125 has received the point analysis acquisition information, the output unit 131 is obtained by processing the member indicated by the member identification information received above with the die indicated by the mold identification information received above. A dialog prompting the user to photograph the processed product from various directions is output to the monitor 120.

  In accordance with this dialog, the user uses a photographing unit 121 of the terminal device 12 to photograph a processed product obtained by processing the member 73 with the mold 71 from a plurality of directions.

  FIG. 10 is a schematic diagram showing a state in which a processed product obtained by processing the member 73 with the mold 71 is photographed. FIGS. 10 (a) to 10 (d) respectively show the processed product. It is a figure which shows the state image | photographed from the left side of 75, the back side, the right side, and the front side. Here, the processed product 75 is photographed from the left side, the rear side, the right side, and the front side, but the photographing direction is not necessarily limited to these directions.

  For example, first, as shown in FIG. 10A, when the user images the processed product 75 from the left side, the imaging unit 121 acquires a captured image (here, it is a still image) and acquires the acquired image A file name including information indicating the shooting order (for example, a serial number indicating the shooting order, shooting time, etc.) is assigned to the image, and the image is stored in the shot image storage unit 122. Here, the imaging processed product shape data acquisition unit 126 determines whether or not the number of captured images stored in the captured image storage unit 122 is equal to or greater than a predetermined number. It is assumed that the designated number of images is, for example, the minimum number of captured images necessary for the imaging processed product shape data acquisition unit 126 to acquire imaging processed product shape data. Here, it is assumed that the specified number is “4”. Here, since there is one stored captured image, it is determined that the number is not greater than the designated number. Therefore, the output unit 131 further outputs a dialog or the like that prompts the user to photograph the processed product. Further, even when the user sequentially photographs as shown in FIGS. 10B and 10C, the photographed images are sequentially stored with file names including information indicating the photographing order. However, since the number of accumulated photographed images does not exceed “4”, a dialog for prompting photographing is similarly output.

  Here, as shown in FIG. 10D, the user images the processed product 75 from the front side, and a file name including information indicating the imaging order is given to the captured image and stored in the captured image storage unit 122. Assume that the imaged processed product shape data acquisition unit 126 counts the number of captured images accumulated in the captured image storage unit 122 and determines that the accumulated captured images are “4” or more.

  Since the number of accumulated captured images is “4” or more, which is the number designated in advance, the imaging processed product shape data acquisition unit 126 uses the four accumulated captured images to process the processed product that is the subject of photographing. The imaging processed product shape data, which is three-dimensional data indicating the three-dimensional shape, is acquired. In addition, since the process which acquires imaging processed product shape data using a some picked-up image is a well-known technique, detailed description is abbreviate | omitted here. The acquired imaging processed product shape data is stored in a storage medium (not shown).

  Further, when the imaging processed product shape data acquisition unit 126 performs processing for acquiring imaging processed product shape data using a plurality of captured images, each captured image is used as a projection view of the imaging processed product shape data to be acquired. In order to handle and acquire imaging processed product shape data, at least temporarily, the imaging processed product shape data is associated with each captured image, so imaging to obtain the same projected image of the processed product shape data as each captured image Values such as the normal vector of the unit 121 and the rotation angle of the imaging unit 121 can be acquired. For this reason, the projection setting information acquisition unit 127 can acquire projection setting information (for example, the normal vector of the imaging unit 121 and information on the rotation angle) during the process of acquiring the imaging processed product shape data. It becomes possible. Then, the acquired projection setting information is stored in a storage medium (not shown) in association with the file name of the latest captured image. Assume that the file name of the latest photographed image is, for example, “img004”.

  When the terminal device 12 has a three-dimensional sensor (not shown) that can detect a three-dimensional direction, such as a three-axis gyro sensor, such as a so-called smartphone, this three-dimensional Using the output of the sensor, the projection setting information acquisition unit 127 acquires a normal vector indicating the shooting direction of the imaging unit 121 and the rotation angle of the imaging unit 121, and projects the normal vector and the rotation angle. It may be used as setting information. In this case, information on the angle of view of the optical system included in the imaging unit 121 may be further acquired. Further, as information on the distance between the imaging unit 121 and the processed product, a fixed value specified in advance may be acquired. In this case, it is preferable for the user to photograph the processed product at a distance as close as possible to the fixed value.

  FIG. 11 is a diagram illustrating the projection setting information acquired by the projection setting information acquisition unit 127, and the projection setting information includes items of “normal vector” and “rotation angle”. The “normal vector” is a vector indicating the line-of-sight direction when rendering from a virtual viewpoint (camera) set to obtain a projection map. “Rotation angle” indicates the rotation angle of this viewpoint with the line-of-sight direction as the rotation axis. Here, for example, the case where the horizontal direction of the visual field viewed from the viewpoint is horizontal is set to 0 degree. This virtual viewpoint may be considered as the imaging unit 121 when the imaging unit 121 images the captured image “img004”. Imaging processed product shape data Imaging processed product shape data

  Next, the analysis processed product shape data acquisition unit 128 uses the point information included in the point analysis acquisition information received and accumulated by the terminal reception unit 125 shown in FIG. Acquire analysis analysis product shape data. Here, as an example, information on meshes and the like indicating a three-dimensional shape are obtained by connecting adjacent point information and the like with straight lines and curves so that the points indicated by the point information are nodes. Here, it is assumed that the analysis processed product shape data includes information on the nodes of the mesh indicating the three-dimensional shape. In addition, since the analysis work product shape data is acquired using the point information, the coordinates of each node are the coordinates indicated by the point information, and each node corresponds to the point information. Point identification information is associated with each node. The analysis processed product shape data acquisition unit 128 accumulates the analysis processed product shape data in a storage medium (not shown).

  Next, the conversion unit 129 performs coordinate conversion of the analysis processed product shape data so that the shape indicated by the analysis processed product shape data overlaps the shape indicated by the imaging processed product shape data.

  FIG. 12 is a diagram for explaining a coordinate conversion process performed by the conversion unit 129. FIG. 12A shows a three-dimensional shape indicated by the imaging processed product shape data as a mesh. FIG. 12B is a diagram showing a three-dimensional shape indicated by the analysis processed product shape data as a mesh. FIG. 12C is a diagram showing the three-dimensional shape indicated by the analysis processed product shape data and the three-dimensional shape indicated by the imaging processed product shape data in the same coordinate system. FIGS. 12D to 12F are diagrams illustrating processing for superimposing the three-dimensional shape indicated by the analysis processed product shape data on the three-dimensional shape indicated by the imaging processed product shape data. In FIG. 12C to FIG. 12F, the shape of the imaging processed product shape data is represented by a dotted line so that the relationship between the shape of the imaging processed product shape data and the shape of the analysis processed product shape data can be easily understood. ing.

  Specifically, the arrangement (position, orientation, size) of the imaging processed product shape data 151 with respect to the virtual three-dimensional coordinate system and the arrangement of the analysis processed product shape data 152 with respect to the virtual three-dimensional coordinate system are as shown in FIG. Usually, as shown in FIG. Therefore, first, the conversion unit 129 detects the first direction with the longest length for both the analysis processed product shape data 152 and the imaging processed product shape data 151 (FIG. 12C). Here, the first direction of the imaging processed product shape data 151 and the first direction of the analysis processed product shape data 152 are indicated by arrows 151a and 152a, respectively. Next, the size of the entire analysis processed product shape data 152 is increased so that the lengths in the first direction of the imaging processed product shape data 151 and the analysis processed product shape data 152 are the same (all in the three-dimensional direction). (Enlarged so that is enlarged at the same magnification) (FIG. 12D).

  Next, the positions of the expanded analysis processed product shape data 152 are placed in a virtual three-dimensional space so that both ends of the expanded analysis processed product shape data 152 in the first direction overlap with both ends of the imaging processed product shape data 152 in the first direction. (Fig. 12 (e)). The movement of the position here may include a rotational movement.

  Furthermore, the direction orthogonal to the first direction and having the longest length (hereinafter referred to as the second direction) is detected for the three-dimensional data indicating the respective shapes. Here, the second direction of the imaging processed product shape data 151 and the second direction of the analytical processed product shape data 152 are indicated by arrows 151b and 152b, respectively. Determining whether or not the intersection of the first direction and the second direction of the analysis processed product shape data 152 coincides with the intersection of the first direction and the second direction of the imaging processed product shape data 151; If they do not match, the analysis processed product shape data 152 is rotated 180 degrees around the midpoint of the first direction of the analyzed processed product shape data 152. Here, since they match, this rotation is not performed.

  Next, virtual three-dimensional analysis data shape data 152 is set to the first direction as a rotation axis so that the second direction of analysis work product shape data 152 overlaps the second direction of imaging work product shape data 151. It is rotated in the space (FIG. 12 (f)). The analysis processed product shape data 152 indicating the shape obtained in this way is the analysis processed product shape data 152 that has been subjected to coordinate conversion, and has a shape that overlaps the shape indicated by the imaging processed product shape data 151. Specifically, in the analysis processed product shape data 152 that has undergone coordinate conversion, the coordinates of the nodes are coordinate-converted. However, the node identification information associated with each node is not changed. The analysis processed product shape data subjected to the coordinate conversion is stored in a storage medium (not shown) or the like.

  Next, the analysis image generation unit 130 converts the coordinate-converted analysis processed product shape data into the projection setting information corresponding to the latest captured image “img004” acquired above and the point analysis acquisition information illustrated in FIG. 9. An analysis image is generated using a set of “point ID” and “analysis information” included. Here, for example, by default, it is assumed that an analysis image indicating an isoline of stress values is first generated, and the analysis image generation unit 130 generates an isoline of stress values. The analysis image shown is generated. First, using “stress” in the “analysis information”, a transfer image relating to stress is generated. Specifically, a set of normal vector and rotation angle is obtained from the projection setting information, and analysis processed product shape data analysis processed product shape data Coordinate-transformed analysis from the viewpoint determined by this normal vector and rotation angle A stress image is generated by rendering the workpiece shape data. In this rendering, the point identification information associated with each node of the mesh of the analysis processed product shape data as described above is read, and “stress” in the “analysis information” associated with each point identification information is read out. Is read out from the point stress management information shown in FIG. 6 as the stress information of the node associated with each point identification information. Then, the analysis image generation unit 130 is a point on the surface of the three-dimensional shape indicated by the coordinate-converted analysis processed product shape data using the stress information for each node, and the value of the stress information is a specific value. Detect a point. The specific value is, for example, one or more values for drawing an isoline when creating an isoline of stress. For example, the specific value is a plurality of discontinuous values. The value interval is a plurality of values at equal intervals. For example, the specific value is set using a range of values indicated by a maximum value and a minimum value of stress. The analysis image generation unit 130 detects a point for each specific value. The point on the surface of the solid shape to be detected may be only a node or may include a point other than the node. When the surface point is a point other than a node, the stress value corresponding to the point is calculated from the coordinates of a plurality of nodes close to the point and the stress values respectively corresponding to the plurality of nodes. . For example, from the coordinates of two adjacent nodes and the stress value of each of these nodes, a relational expression (for example, approximate expression) indicating the relationship between the coordinates of the two nodes and the stress value is obtained. From this relational expression, the coordinates of the point where the stress value is a specific value may be calculated. Further, for example, (α + β) / 2 may be acquired as the stress value at the midpoint between the node A having the stress value α and the node B having the stress value β.

  Then, the analysis image generation unit 130 places one or more isolines connecting a plurality of points on the surface of the analytical work product shape data whose stress value is a specific common value on the surface of the analytical work product shape data. The analyzed image is generated by rendering with the setting indicated by the projection setting information. As a result, an analysis image that is a projection view of the analytical work product shape data in which the stress isolines are arranged on the surface is generated. Each isoline may have a different color for each corresponding value. Further, here, as an example, an analysis image in which a portion other than the isoline is transparent is generated. Further, here, an analysis image in which an image or a character string indicating the maximum and minimum values of stress is further arranged at the position indicated by the node indicating the maximum value of stress and the position indicated by the node indicating the minimum value Is generated. Further, information such as a legend representing the value indicated by each isoline is also arranged in the analysis image.

  FIG. 13 is a diagram illustrating an analysis image of stress generated by the stress image generation unit 130.

  Next, the output unit 131 generates and outputs an image obtained by superimposing the stress analysis image generated by the stress image generation unit 130 on the captured image “img004”. Here, the output unit 131 displays on the monitor 120.

  FIG. 14 is a diagram illustrating an output example of an image in which an analysis image of stress is superimposed on a captured image. Thereby, an image in which an image showing a stress distribution (that is, showing an isoline of stress) is superimposed on the latest photographed image can be output. As a result, it is possible to display an analysis result related to stress on an image obtained by photographing an actual processed product.

  In addition, the user gives an instruction to output an image in which the analysis image indicating the analysis result of the plate thickness is superimposed on the captured image instead of the analysis image indicating the analysis result of the stress on the captured image via a reception unit (not illustrated). If given, the analysis image generation unit 130 uses the value of “plate thickness” instead of the value of “stress” of “analysis information” included in the point analysis acquisition information, and similarly to the above, An analysis image showing an isoline of values is generated. Then, the output unit 131 outputs the analysis image so as to overlap the photographed image.

  Similarly, if the user gives an instruction to output an image obtained by superimposing an analysis image indicating the analysis result of distortion on a captured image via a reception unit (not shown) or the like, the analysis image generation unit 130 includes point analysis acquisition information. Using the “distortion” value of the “analysis information” included in the image, an analysis image indicating an isoline of the distortion value is generated in the same manner as described above. Then, the output unit 131 outputs the analysis image so as to overlap the photographed image.

  Here, for example, in the state shown in FIG. 14, if the user further photographs the processed product 75 from a different direction using the imaging unit 121, the imaging unit 121 captures the captured image. Are stored in the photographed image storage unit 122.

  Since the captured image storage unit 122 stores four or more captured images, using the captured image including the latest image, the imaging processed product shape data acquisition unit 126 is similar to the above. The imaged processed product shape data is acquired, and the projection setting information acquisition unit 127 acquires projection setting information corresponding to the latest photographed image. If the projection setting information can be acquired, the imaging processed product shape data may not be acquired. For example, the projection setting information acquisition unit 127 may perform the process of acquiring the imaging processed product shape data until the projection setting information is obtained, and acquire the projection setting information.

  Here, since the coordinate conversion of the analysis processed product shape data has already been performed by the above processing, the analysis image generation unit 130 corresponds to the analysis processed product shape data subjected to the coordinate conversion and the latest photographed image. Using the projection setting information and the analysis information associated with the point identification information, an analysis image is generated in the same manner as described above. Then, the output unit 131 outputs the analysis information superimposed on the latest photographed image. Thereby, whenever the imaging | photography part 121 image | photographs a processed product, it becomes possible to output the information regarding the analysis result acquired by analysis to the picked-up image obtained by image | photographing in real time.

  As described above, in the present embodiment, the analysis result obtained by performing analysis or the like on the combination of the mold and the member used for processing the processed product on the image of the processed product photographed by the terminal device 12. The information can be output in an overlapping manner, and the relationship between the actual processed product and the analysis result can be easily output to the user regardless of where the user is.

  In the above embodiment, the point analysis information storage unit 110 includes mold identification information, member identification information of a member processed using the mold indicated by the mold identification information, and one or more point information. Point analysis information having a set of and analysis information is stored. However, in the present invention, for example, when only one member is processed by the mold, or analysis of the mold acquired for the one member is the same mold regardless of the difference in the members. In the case of using as an analysis result (or reference value) of an analysis of a processed product processed by using, for example, it is not necessary for the acquisition unit 113 to acquire analysis information or the like according to the processed member. As the information, point analysis information in which member identification information is omitted, that is, point analysis information having mold identification information and a set of one or more pieces of point information and analysis information may be used. In this case, for example, the server reception unit 112 receives at least the mold identification information from the terminal device 11, and the acquisition unit 113 receives the point information corresponding to the mold identification information received by the server reception unit 112 and the analysis information. The set is acquired from the point analysis information, and the reception unit 123 receives at least the mold identification information of the mold used for processing the processed product photographed by the imaging unit 121, and the terminal transmission unit 124 receives the reception unit The mold identification information received by 123 may be transmitted to the server device 11. The same applies to the following embodiments.

(Embodiment 2)
In the second embodiment, the server device 11 and the terminal device 12 of the first embodiment are realized by one information processing device.

  FIG. 15 is a block diagram illustrating a configuration of the information processing apparatus 20 according to the second embodiment. In the figure, the same reference numerals as those in FIG. 1 denote the same or corresponding parts.

  The acquisition unit 213 obtains one or more sets of one or more pieces of point information corresponding to the mold identification information and member identification information received by the reception unit 123 and analysis information corresponding to each piece of point information, as point analysis information. Obtained from the storage unit 110. Since other parts are the same as those of the acquisition unit 113, description thereof is omitted. The point information and the analysis information acquired by the acquisition unit 213 are, for example, the analysis processed product shape data acquisition unit 128 instead of the point information and the analysis information received by the terminal reception unit 125 of the first embodiment. Etc.

  Next, the operation of the information processing apparatus of this embodiment will be described using the flowchart of FIG. In FIG. 16, the same reference numerals as those in FIGS. 2 and 3 denote the same or corresponding steps.

  (Step S1601) The acquisition unit 113 sets one or more sets of one or more pieces of point information corresponding to the mold identification information and the member identification information received by the reception unit 123 and analysis information corresponding to each piece of point information. , Acquired from the point analysis information storage unit 110. Then, the process proceeds to step S205.

  (Step S1602) The reception unit 123 determines whether an instruction to end the output of an image obtained by superimposing the captured image and the analysis image has been received. If accepted, the output unit 131 ends the output, and the process proceeds to step S101. If not accepted, the process returns to step S206.

  In the flowchart of FIG. 16, the process is terminated by power-off or a process termination interrupt.

  Also in the information processing apparatus 20 of the present embodiment, similarly to the information processing system described above, information related to an analysis result acquired in advance by analysis or the like is output on an image obtained by photographing a processed product. Can do. In addition, unlike the information processing system described above, it is not necessary to connect the server device 11 and the terminal device 12 via a network, so the information processing device 20 can be used even in a place without a network environment. .

(Embodiment 3)
In the first embodiment, the case where the point stress information for each mold identification information is stored in advance in the point stress information storage unit 110 has been described. However, in the present embodiment, the server reception unit 112 has the gold Type identification information,
FIG. 19 is a diagram showing a configuration of the information processing system 2 according to Embodiment 3 of the present invention. The information processing system 2 is configured such that a server device 21 is provided instead of the server device 11 in the information processing system according to the first embodiment.
The server device 21 includes a mold three-dimensional data storage unit 101, a member data storage unit 102, a point analysis information storage unit 110, a point analysis information acquisition unit 111, a server reception unit 112, an acquisition unit 113, and a server transmission unit 114. Yes. Since the point analysis information storage unit 110, the point analysis information acquisition unit 111, the server reception unit 112, the acquisition unit 113, and the server transmission unit 114 are the same as those in the first embodiment, description thereof will be omitted.
The mold 3D data storage unit 101 stores mold management information including mold 3D data and mold identification information. The mold three-dimensional data is three-dimensional data indicating the three-dimensional shape of the mold as described above, and is, for example, mold mesh data.
The member data storage unit 102 stores member management information having member data and member identification information. The member data is information regarding the member to be processed as described above, and is data such as material characteristics of the member.
Here, the point analysis information acquisition unit 111, when the server reception unit 112 receives the mold identification information or when receiving the mold identification information, the mold tertiary corresponding to the mold identification information. Original data is acquired from the mold 3D data storage unit 101. Moreover, when member identification information is acquired in addition to mold identification information, the member data storage unit 102 further acquires member data corresponding to the member identification information. Then, the point analysis information is acquired as described in the first embodiment using the acquired mold three-dimensional data (or the mold three-dimensional data and member data). The acquired point analysis information is temporarily stored in the point analysis information storage unit 110.
Since the configuration and operation of the terminal device 12 are the same as those in the first embodiment, description thereof is omitted here.
Next, an example of operation | movement of the server apparatus 1 of this Embodiment is demonstrated using the flowchart of FIG.

  (Step S2001) The server reception unit 112 determines whether or not the mold identification information and the member identification information are received from the one terminal device 12. If received, the process proceeds to step S102. If not received, the process returns to step S101.

  (Step S2002) The point analysis information acquisition unit 111 sets a combination of the mold three-dimensional data corresponding to the mold identification information received in step S2001 and the member data corresponding to the member identification information received in step S2001 to the mold. Obtained from the mold three-dimensional data storage unit 101 and the member data storage unit 102, respectively.

  (Step S2003) The point analysis information acquisition unit 111 performs a simulation or the like for performing a predesignated analysis using the mold three-dimensional data acquired in step S2002 and the member data, and the member data is acquired by the mold. A set of point information indicating one or more points on the surface of the three-dimensional shape obtained when processing and analysis information at the points indicated by each point information is acquired.

  (Step S2004) The point analysis information acquisition unit 111 has one or more sets of the point information and analysis information acquired in step S2003, the mold identification information received in step S2001, and the member analysis information. Is temporarily stored in the point analysis information storage unit 110.

  (Step S2005) The acquisition unit 113 reads one or more sets of point information and analysis information corresponding to the set of mold identification information and member identification information temporarily stored in Step S2004.

  (Step S2006) The server-side transmitting unit 114 is a terminal device that is a transmission source of the mold identification information and the member identification information received in step S2001 for one or more pairs of the point information and analysis information read in step S2005. 12 to send. Then, the process returns to step S2001.

  In the flowchart of FIG. 20, the process is terminated by powering off or a process termination interrupt.

  In the present embodiment, the same effect as in the first embodiment is obtained, and in the present embodiment, the server device 21 receives the analysis information and the like when the mold identification information and the like are transmitted from the terminal device 12. Since it is not necessary to prepare analysis information in advance on the server device 11 side in order to obtain the information, unnecessary analysis processing can be prevented and a storage medium for storing analysis information can be minimized. There is an effect that it can be suppressed to the limit.

  In each of the above embodiments, each process (each function) may be realized by centralized processing by a single device (system), or by distributed processing by a plurality of devices. May be.

  Further, in each of the above embodiments, it goes without saying that two or more communication means (such as an information transmission unit) existing in one apparatus may be physically realized by one medium.

  In each of the above embodiments, the point analysis information acquisition unit 111, the acquisition unit 113, the imaging processed product shape data acquisition unit 126, the projection setting information acquisition unit 127, the analysis processed product shape data acquisition unit 128, the conversion unit 129, the analysis Each component such as the image generation unit 130 and the acquisition unit 213 may be configured by dedicated hardware, or a component that can be realized by software may be realized by executing a program. For example, each component can be realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. At the time of execution, the program execution unit may execute the program while accessing a storage unit (for example, a recording medium such as a hard disk or a memory).

  The software that realizes the server device in each of the above embodiments is a program as described below. That is, this program is information indicating mold identification information which is identification information of the mold and one or more points on the surface of the processed member when the member is processed into a shape corresponding to the mold. One or more point information and information obtained by performing a pre-designated analysis on the processed member, each point indicated by one or more point information on the surface of the processed member A server that receives a mold identification information from a terminal device, a computer that can access a point analysis information storage unit that stores one or more point analysis information having at least one analysis information that is an analysis result of And an acquisition unit for acquiring one or more sets of point information corresponding to the mold identification information received by the server reception unit and analysis information corresponding to each point information from the point analysis information storage unit; Point information and solution acquired by the acquisition unit A server transmitter for transmitting one or more sets of the information to the terminal device, a program for causing to function.

  In addition, the software that realizes the terminal device in each of the above embodiments includes an imaging unit that acquires one or more captured images that are images obtained by capturing a processed product obtained by processing a computer using a mold on a computer. A reception unit that receives mold identification information of a mold used for processing the processed product photographed by the imaging unit, a terminal transmission unit that transmits the mold identification information received by the reception unit to the server device, and point information An imaging processed product shape, which is three-dimensional data indicating the shape of a processed product that has been imaged using a terminal receiving unit that receives one or more sets of analysis information from the server device and one or more captured images An image projection product shape data acquisition unit for acquiring data, and a projection diagram in which the shape indicated by the image pickup work product shape data is projected with respect to each of one or more of the captured images used when acquiring the image pickup work product shape data. obtain Information indicating a setting for obtaining a projection drawing in which the shape indicated by the imaged workpiece shape data in the projection drawing is the same as the shape of the workpiece in the photographed image. Projection setting information acquisition unit for acquiring projection setting information, and analysis workpiece shape data for acquiring three-dimensional data representing the shape corresponding to the mold indicated by the mold identification information received by the reception unit A data acquisition unit, a conversion unit that converts the analysis processed product shape data so that the analysis processed product shape data overlaps the imaging processed product shape data, an analysis processed product shape data that is converted by the conversion unit, and an analysis processed product shape Using the analysis information corresponding to point information indicating one or more points on the surface of the three-dimensional shape indicated by the data and one of the projection setting information, the analysis information is indicated on the surface of the shape indicated by the analysis work product shape data. An analysis image generation unit that generates an analysis image that is an image of a projection diagram showing information related to the analysis result in accordance with the setting indicated by the projection setting information, and a captured image corresponding to the projection setting information used by the analysis image generation unit; It is a program for functioning as an output unit that outputs an image obtained by superimposing an analysis image.

  Moreover, the software which implement | achieves the information processing apparatus in each said embodiment is the surface of the processed member at the time of processing the metal mold identification information which is metal mold identification information, and a member into the shape corresponding to a metal mold | die. One or more point information that is information indicating one or more points, and information obtained by performing a predesignated analysis on the processed member, the surface of the processed member A computer capable of accessing a point analysis information storage unit in which one or more point analysis information having one or more pieces of point analysis information having one or more pieces of analysis information as information of analysis results for each point indicated by the one or more point information An imaging unit that acquires one or more captured images that are images obtained by processing a member obtained by processing a member using a mold, and a mold identification of a mold used for processing the processed product captured by the imaging unit Reception unit for receiving information and reception unit An acquisition unit that acquires one or more sets of one or more point information corresponding to the received mold identification information and analysis information corresponding to each point information from the point analysis information storage unit, and one or more captured images Used to acquire imaging processed product shape data, and an imaging processed product shape data acquisition unit that acquires imaging processed product shape data, which is three-dimensional data indicating the shape of the processed product to be imaged. For each of one or more of the captured images, information indicating a setting for obtaining a projection diagram in which the shape indicated by the imaging processed product shape data is projected, and the shape indicated by the imaging processed product shape data in the projection diagram is: A projection setting information acquisition unit that acquires projection setting information, which is information indicating settings for obtaining a projection that has the same shape as the processed product in the captured image, and a mold indicated by the mold identification information received by the reception unit Vs type The analysis work product shape data acquisition unit that obtains the analysis work product shape data that is three-dimensional data indicating the shape to be processed, and the coordinate conversion of the analysis work product shape data so that the analysis work product shape data overlaps the imaging work product shape data A conversion unit that performs coordinate conversion by the conversion unit, analysis information corresponding to point information indicating one or more points on the surface of the three-dimensional shape indicated by the analysis processed product shape data, and projection setting information Analysis that generates an analysis image, which is an image of a projection diagram showing information related to the analysis result indicated by the analysis information, on the surface of the shape indicated by the analysis processed product shape data according to the setting indicated by the projection setting information This is a program for functioning as an image generation unit and an output unit that outputs an image obtained by superimposing a captured image and an analysis image corresponding to the projection setting information used by the analysis image generation unit.

  In the program, the functions realized by the program do not include functions that can be realized only by hardware. For example, a function that can be realized only by hardware such as a modem or an interface card in an acquisition unit that acquires information or an output unit that outputs information is not included in the function realized by the program.

  Further, the computer that executes this program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  FIG. 17 is a schematic diagram illustrating an example of the appearance of a computer that executes the program and realizes the information processing system according to the embodiment. The above-described embodiment can be realized by computer hardware and a computer program executed on the computer hardware.

  In FIG. 17, a computer system 900 includes a computer 901 including a CD-ROM (Compact Disk Read Only Memory) drive 905, an FD (Floppy (registered trademark) Disk) drive 906, a keyboard 902, a mouse 903, a monitor 904, and the like. Is provided.

  FIG. 18 is a diagram showing an internal configuration of the computer system 900. In FIG. 18, in addition to the CD-ROM drive 905 and the FD drive 906, a computer 901 is connected to an MPU (Micro Processing Unit) 911, a ROM 912 for storing a program such as a bootup program, and the MPU 911. A RAM (Random Access Memory) 913 that temporarily stores program instructions and provides a temporary storage space, a hard disk 914 that stores application programs, system programs, and data, and an MPU 911 and a ROM 912 are interconnected. And a bus 915. The computer 901 may include a network card (not shown) that provides connection to the LAN.

  A program that causes the computer system 900 to execute functions such as the information processing system according to the above embodiment is stored in the CD-ROM 921 or the FD 922, inserted into the CD-ROM drive 905 or the FD drive 906, and stored in the hard disk 914. May be forwarded. Instead, the program may be transmitted to the computer 901 via a network (not shown) and stored in the hard disk 914. The program is loaded into the RAM 913 when executed. The program may be loaded directly from the CD-ROM 921, the FD 922, or the network.

  The program does not necessarily include an operating system (OS) or a third-party program that causes the computer 901 to execute the functions of the information processing system according to the above embodiment. The program may include only a part of an instruction that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 900 operates is well known and will not be described in detail.

  The present invention is not limited to the above-described embodiments, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, the information processing system and the like according to the present invention are suitable as an apparatus for outputting information on a processed product processed using a mold, and in particular, this processing is performed on an image obtained by photographing the processed product. It is useful as a device that outputs information on products in a superimposed manner.

DESCRIPTION OF SYMBOLS 1 Information processing system 11 Server apparatus 12 Terminal apparatus 20 Information processing apparatus 70 Press molding machine 71 Mold 72, 74 Two-dimensional barcode 73 Member 75 Workpiece 101 Mold three-dimensional data storage part 102 Member data storage part 110 Point analysis Information storage unit 111 Point analysis information acquisition unit 112 Server reception unit 113, 213 acquisition unit 114 Server transmission unit 120 Monitor 121 Imaging unit 122 Captured image storage unit 123 Reception unit 124 Terminal transmission unit 125 Terminal reception unit 126 Image processing product shape data acquisition Unit 127 projection setting information acquisition unit 128 analysis processed product shape data acquisition unit 129 conversion unit 130 analysis image generation unit 131 output unit 151 imaging processed product shape data 152 analysis processed product shape data

Claims (12)

An information processing system having a server device and a terminal device,
The server device
Mold identification information which is identification information of a mold and one or more points which are information indicating one or more points on the surface of the processed member when the member is processed into a shape corresponding to the mold Information and information obtained by performing a predesignated analysis on the processed member, and analyzing each point indicated by the one or more point information on the surface of the processed member A point analysis information storage unit that stores one or more pieces of point analysis information having one or more pieces of analysis information that is information indicating results;
A server receiving unit for receiving mold identification information from the terminal device;
An acquisition unit that acquires, from the point analysis information storage unit, one or more sets of one or more point information corresponding to the mold identification information received by the server reception unit and analysis information corresponding to each point information; ,
A server transmission unit that transmits one or more sets of point information and analysis information acquired by the acquisition unit to the terminal device;
The terminal device
An imaging unit that obtains one or more captured images that are images of a processed product obtained by processing a member using a mold;
A receiving unit for receiving mold identification information of a mold used for processing a processed product photographed by the imaging unit;
A terminal transmission unit that transmits the mold identification information received by the reception unit to the server device;
A terminal receiver that receives one or more sets of point information and analysis information from the server device;
An imaging processed product shape data acquisition unit that acquires imaging processed product shape data, which is three-dimensional data indicating the shape of a processed product that has been imaged, using one or more of the captured images;
Information indicating a setting for obtaining a projected image obtained by projecting the shape indicated by the imaging processed product shape data for each of one or more of the captured images used when acquiring the imaging processed product shape data, A projection setting information acquisition unit that acquires projection setting information, which is information indicating settings for obtaining a projection diagram in which the shape indicated by the imaged workpiece shape data in the projection diagram is the same as the shape of the workpiece in the captured image When,
An analysis processed product shape data acquisition unit that acquires analytical processed product shape data that is three-dimensional data indicating a shape corresponding to the mold indicated by the mold identification information received by the reception unit;
A conversion unit that performs coordinate conversion of the analysis processed product shape data so that the analyzed processed product shape data overlaps with the imaging processed product shape data;
One piece of analysis work product shape data coordinate-converted by the conversion unit, the analysis information corresponding to point information indicating one or more points on the surface of the three-dimensional shape indicated by the analysis work product shape data, and one of the projection setting information An analysis image that is an image of a projection diagram showing information on the analysis result indicated by the analysis information on the surface of the shape indicated by the analysis processed product shape data according to the setting indicated by the projection setting information. An analysis image generation unit to perform,
An information processing system comprising: an output unit that outputs an image obtained by superimposing a captured image corresponding to the projection setting information used by the analysis image generation unit and the analysis image. The point analysis information further includes member identification information that is information for identifying a member to be processed using a mold indicated by the mold identification information,
The server reception unit receives a set of mold identification information and member identification information from the terminal device,
The acquisition unit obtains one or more sets of one or more pieces of point information corresponding to the mold identification information and the member identification information received by the server reception unit and analysis information corresponding to each piece of point information. Obtained from the analysis information storage,
The reception unit further receives member identification information of a member used for the processing,
The information processing system according to claim 1, wherein the terminal transmission unit transmits the mold identification information and the member identification information received by the reception unit to the server device. The imaging processed product shape data acquisition unit acquires imaging processed product shape data every time the imaging unit acquires one captured image,
The projection setting information acquisition unit acquires the projection setting information every time the imaging unit acquires one captured image,
The analysis image generation unit generates an analysis image using the projection setting information acquired by the projection setting information acquisition unit for the one captured image,
The information processing system according to claim 1, wherein the output unit outputs an image obtained by superimposing the one captured image and the analysis image every time the imaging unit acquires one captured image. Mold identification information which is identification information of a mold and one or more points which are information indicating one or more points on the surface of the processed member when the member is processed into a shape corresponding to the mold Information and information obtained by performing a predesignated analysis on the processed member, and analyzing each point indicated by the one or more point information on the surface of the processed member A point analysis information storage unit that stores one or more pieces of point analysis information having one or more pieces of analysis information that is information indicating results;
A server receiver that receives mold identification information from the terminal device;
An acquisition unit that acquires, from the point analysis information storage unit, one or more sets of one or more point information corresponding to the mold identification information received by the server reception unit and analysis information corresponding to each point information; ,
The server apparatus provided with the server transmission part which transmits 1 or more sets of the point information and analysis information which the said acquisition part acquired to the said terminal device. An imaging unit that obtains one or more captured images that are images of a processed product obtained by processing a member using a mold;
A receiving unit for receiving mold identification information of a mold used for processing a processed product photographed by the imaging unit;
A terminal transmission unit that transmits the mold identification information received by the reception unit to the server device;
Point information that is information indicating one or more points on the surface of the processed member when the member is processed into a shape corresponding to the mold indicated by the mold identification information transmitted by the terminal transmission unit ; Information obtained by performing a predesignated analysis on the processed member, and information indicating an analysis result for each point indicated by the one or more point information on the surface of the processed member a terminal receiving portion of one or more sets of the analysis information received from the server apparatus is,
An imaging processed product shape data acquisition unit that acquires imaging processed product shape data, which is three-dimensional data indicating the shape of a processed product that has been imaged, using one or more of the captured images;
Information indicating a setting for obtaining a projected image obtained by projecting the shape indicated by the imaging processed product shape data for each of one or more of the captured images used when acquiring the imaging processed product shape data, A projection setting information acquisition unit that acquires projection setting information, which is information indicating settings for obtaining a projection diagram in which the shape indicated by the imaged workpiece shape data in the projection diagram is the same as the shape of the workpiece in the captured image When,
An analysis processed product shape data acquisition unit that acquires analytical processed product shape data that is three-dimensional data indicating a shape corresponding to the mold indicated by the mold identification information received by the reception unit;
A conversion unit that performs coordinate conversion of the analysis processed product shape data so that the analyzed processed product shape data overlaps with the imaging processed product shape data;
One piece of analysis work product shape data coordinate-converted by the conversion unit, the analysis information corresponding to point information indicating one or more points on the surface of the three-dimensional shape indicated by the analysis work product shape data, and one of the projection setting information An analysis image that is an image of a projection diagram showing information on the analysis result indicated by the analysis information on the surface of the shape indicated by the analysis processed product shape data according to the setting indicated by the projection setting information. An analysis image generation unit to perform,
A terminal device comprising: an output unit that outputs an image obtained by superimposing a captured image corresponding to the projection setting information used by the analysis image generation unit and the analysis image. Mold identification information which is identification information of a mold and one or more points which are information indicating one or more points on the surface of the processed member when the member is processed into a shape corresponding to the mold Information and information obtained by performing a predesignated analysis on the processed member, and analyzing each point indicated by the one or more point information on the surface of the processed member A point analysis information storage unit that stores one or more pieces of point analysis information having one or more pieces of analysis information that is information indicating results;
An imaging unit that obtains one or more captured images that are images of a processed product obtained by processing a member using a mold;
A receiving unit for receiving mold identification information of a mold used for processing a processed product photographed by the imaging unit;
An acquisition unit that acquires, from the point analysis information storage unit, one or more sets of one or more point information corresponding to the mold identification information received by the reception unit and analysis information corresponding to the point information;
An imaging processed product shape data acquisition unit that acquires imaging processed product shape data, which is three-dimensional data indicating the shape of a processed product that has been imaged, using one or more of the captured images;
Information indicating a setting for obtaining a projected image obtained by projecting the shape indicated by the imaging processed product shape data for each of one or more of the captured images used when acquiring the imaging processed product shape data, A projection setting information acquisition unit that acquires projection setting information, which is information indicating settings for obtaining a projection diagram in which the shape indicated by the imaged workpiece shape data in the projection diagram is the same as the shape of the workpiece in the captured image When,
An analysis processed product shape data acquisition unit that acquires analytical processed product shape data that is three-dimensional data indicating a shape corresponding to the mold indicated by the mold identification information received by the reception unit;
A conversion unit that performs coordinate conversion of the analysis processed product shape data so that the analyzed processed product shape data overlaps with the imaging processed product shape data;
One piece of analysis work product shape data coordinate-converted by the conversion unit, the analysis information corresponding to point information indicating one or more points on the surface of the three-dimensional shape indicated by the analysis work product shape data, and one of the projection setting information An analysis image that is an image of a projection diagram showing information on the analysis result indicated by the analysis information on the surface of the shape indicated by the analysis processed product shape data according to the setting indicated by the projection setting information. An analysis image generation unit to perform,
An information processing apparatus comprising: an output unit that outputs an image obtained by superimposing a captured image corresponding to the projection setting information used by the analysis image generation unit and the analysis image. Mold identification information which is identification information of a mold and one or more points which are information indicating one or more points on the surface of the processed member when the member is processed into a shape corresponding to the mold Information and information obtained by performing a predesignated analysis on the processed member, and analyzing each point indicated by the one or more point information on the surface of the processed member Information performed by using a point analysis information storage unit, a server reception unit, an acquisition unit, and a server transmission unit in which one or more point analysis information having one or more pieces of analysis information indicating information is stored A processing method,
A server receiving step in which the server receiving unit receives mold identification information from a terminal device;
The acquisition unit obtains one or more sets of one or more pieces of point information corresponding to the mold identification information received in the server reception step and analysis information corresponding to the point information from the point analysis information storage unit. An acquisition step to acquire;
An information processing method comprising: a server transmission step in which the server transmission unit transmits one or more sets of point information and analysis information acquired in the acquisition step to the terminal device. An imaging unit, a reception unit, a terminal side transmission unit, a terminal side reception unit, an imaging processed product shape data acquisition unit, a projection setting information acquisition unit, an analysis processed product shape data acquisition unit, a conversion unit, and an analysis An information processing method performed using an image generation unit and an output unit,
An imaging step in which the imaging unit acquires one or more captured images that are images of a processed product obtained by processing a member using a mold;
A receiving step for receiving mold identification information of a mold used for processing the processed product photographed in the imaging step;
A terminal transmission step in which the terminal-side transmission unit transmits the mold identification information received in the reception step to the server device;
Information indicating one or more points on the surface of the processed member when the terminal receiving unit processes the member into a shape corresponding to the mold indicated by the mold identification information transmitted in the terminal transmission step. A point information and information obtained by performing a predetermined analysis on the processed member, and each point indicated by the one or more point information on the surface of the processed member A terminal receiving step of receiving from the server device one or more sets of analysis information which is information indicating the analysis result of
Imaging processed product shape data for acquiring imaging processed product shape data, which is three-dimensional data indicating the shape of the processed product to be imaged, using the one or more captured images. An acquisition step;
Setting for obtaining a projection in which the projection setting information acquisition unit projects the shape indicated by the imaging processed product shape data with respect to each of one or more of the captured images used when acquiring the imaging processed product shape data Projection setting information, which is information indicating the setting for obtaining a projection diagram in which the shape indicated by the imaged workpiece shape data in the projection image is the same as the shape of the workpiece in the captured image A projection setting information acquisition step for acquiring
The analytical processed product shape data acquisition unit acquires analytical processed product shape data that is three-dimensional data indicating the shape corresponding to the mold indicated by the mold identification information received in the receiving step. Steps,
A conversion step in which the conversion unit coordinates-transforms the analysis processed product shape data so that the analyzed processed product shape data overlaps with the imaging processed product shape data;
The analytical image corresponding to the point information indicating one or more points on the surface of the three-dimensional shape indicated by the analysis processed product shape data, and the analysis processed product shape data coordinate-converted in the conversion step by the analysis image generation unit; , Using the one of the projection setting information, an analysis image that is an image of a projection diagram showing information on the analysis result indicated by the analysis information on the surface of the shape indicated by the analysis processed product shape data, and the projection setting information An analysis image generation step generated according to the setting indicated by
An information processing method comprising: an output step in which the output unit outputs an image obtained by superimposing the captured image corresponding to the projection setting information used in the analysis image generation step and the analysis image. Mold identification information which is identification information of a mold and one or more points which are information indicating one or more points on the surface of the processed member when the member is processed into a shape corresponding to the mold Information and information obtained by performing a predesignated analysis on the processed member, and analyzing each point indicated by the one or more point information on the surface of the processed member A point analysis information storage unit that stores one or more point analysis information having one or more pieces of analysis information indicating information, an imaging unit, a reception unit, an acquisition unit, and an imaging processed product shape data acquisition unit And an information processing method performed using a projection setting information acquisition unit, an analysis processed product shape data acquisition unit, a conversion unit, an analysis image generation unit, and an output unit,
An imaging step in which the imaging unit acquires one or more captured images that are images of a processed product obtained by processing a member using a mold;
A receiving step for receiving mold identification information of a mold used for processing the processed product photographed in the imaging step;
The acquisition unit acquires, from the point analysis information storage unit, one or more sets of one or more point information corresponding to the mold identification information received in the reception step and analysis information corresponding to the point information. An acquisition step to
Imaging processed product shape data for acquiring imaging processed product shape data, which is three-dimensional data indicating the shape of the processed product to be imaged, using the one or more captured images. An acquisition step;
Setting for obtaining a projection in which the projection setting information acquisition unit projects the shape indicated by the imaging processed product shape data with respect to each of one or more of the captured images used when acquiring the imaging processed product shape data Projection setting information, which is information indicating the setting for obtaining a projection diagram in which the shape indicated by the imaged workpiece shape data in the projection image is the same as the shape of the workpiece in the captured image A projection setting information acquisition step for acquiring
The analytical processed product shape data acquisition unit acquires analytical processed product shape data that is three-dimensional data indicating the shape corresponding to the mold indicated by the mold identification information received in the receiving step. Steps,
A conversion step in which the conversion unit coordinates-transforms the analysis processed product shape data so that the analyzed processed product shape data overlaps with the imaging processed product shape data;
The analytical image corresponding to the point information indicating one or more points on the surface of the three-dimensional shape indicated by the analysis processed product shape data, and the analysis processed product shape data coordinate-converted in the conversion step by the analysis image generation unit; , Using the one of the projection setting information, an analysis image that is an image of a projection diagram showing information on the analysis result indicated by the analysis information on the surface of the shape indicated by the analysis processed product shape data, and the projection setting information An analysis image generation step generated according to the setting indicated by
An information processing method comprising: an output step in which the output unit outputs an image obtained by superimposing the captured image corresponding to the projection setting information used in the analysis image generation step and the analysis image. Mold identification information which is identification information of a mold and one or more points which are information indicating one or more points on the surface of the processed member when the member is processed into a shape corresponding to the mold Information and information obtained by performing a predesignated analysis on the processed member, and analyzing each point indicated by the one or more point information on the surface of the processed member A computer capable of accessing a point analysis information storage unit in which one or more point analysis information having one or more pieces of analysis information indicating information is stored;
A server receiver that receives mold identification information from the terminal device;
An acquisition unit that acquires, from the point analysis information storage unit, one or more sets of one or more point information corresponding to the mold identification information received by the server reception unit and analysis information corresponding to each point information; ,
A program for functioning as a server transmission unit that transmits one or more sets of point information and analysis information acquired by the acquisition unit to the terminal device. Computer
An imaging unit that obtains one or more captured images that are images of a processed product obtained by processing a member using a mold;
A receiving unit for receiving mold identification information of a mold used for processing a processed product photographed by the imaging unit;
A terminal transmission unit that transmits the mold identification information received by the reception unit to the server device;
Point information that is information indicating one or more points on the surface of the processed member when the member is processed into a shape corresponding to the mold indicated by the mold identification information transmitted by the terminal transmission unit ; Information obtained by performing a predesignated analysis on the processed member, and information indicating an analysis result for each point indicated by the one or more point information on the surface of the processed member a terminal receiving portion of one or more sets of the analysis information received from the server apparatus is,
An imaging processed product shape data acquisition unit that acquires imaging processed product shape data, which is three-dimensional data indicating the shape of a processed product that has been imaged, using one or more of the captured images;
Information indicating a setting for obtaining a projected image obtained by projecting the shape indicated by the imaging processed product shape data for each of one or more of the captured images used when acquiring the imaging processed product shape data, A projection setting information acquisition unit that acquires projection setting information, which is information indicating settings for obtaining a projection diagram in which the shape indicated by the imaged workpiece shape data in the projection diagram is the same as the shape of the workpiece in the captured image When,
An analysis processed product shape data acquisition unit that acquires analytical processed product shape data that is three-dimensional data indicating a shape corresponding to the mold indicated by the mold identification information received by the reception unit;
A conversion unit that performs coordinate conversion of the analysis processed product shape data so that the analyzed processed product shape data overlaps with the imaging processed product shape data;
One piece of analysis work product shape data coordinate-converted by the conversion unit, the analysis information corresponding to point information indicating one or more points on the surface of the three-dimensional shape indicated by the analysis work product shape data, and one of the projection setting information An analysis image that is an image of a projection diagram showing information on the analysis result indicated by the analysis information on the surface of the shape indicated by the analysis processed product shape data according to the setting indicated by the projection setting information. An analysis image generation unit to perform,
A program for functioning as an output unit that outputs an image obtained by superimposing a captured image corresponding to the projection setting information used by the analysis image generation unit and the analysis image. Mold identification information which is identification information of a mold and one or more points which are information indicating one or more points on the surface of the processed member when the member is processed into a shape corresponding to the mold Information and information obtained by performing a predesignated analysis on the processed member, and analyzing each point indicated by the one or more point information on the surface of the processed member A computer capable of accessing a point analysis information storage unit in which one or more point analysis information having one or more pieces of analysis information indicating information is stored;
An imaging unit that obtains one or more captured images that are images of a processed product obtained by processing a member using a mold;
A receiving unit for receiving mold identification information of a mold used for processing a processed product photographed by the imaging unit;
An acquisition unit that acquires, from the point analysis information storage unit, one or more sets of one or more point information corresponding to the mold identification information received by the reception unit and analysis information corresponding to the point information;
An imaging processed product shape data acquisition unit that acquires imaging processed product shape data, which is three-dimensional data indicating the shape of a processed product that has been imaged, using one or more of the captured images;
Information indicating a setting for obtaining a projected image obtained by projecting the shape indicated by the imaging processed product shape data for each of one or more of the captured images used when acquiring the imaging processed product shape data, A projection setting information acquisition unit that acquires projection setting information, which is information indicating settings for obtaining a projection diagram in which the shape indicated by the imaged workpiece shape data in the projection diagram is the same as the shape of the workpiece in the captured image When,
An analysis processed product shape data acquisition unit that acquires analytical processed product shape data that is three-dimensional data indicating a shape corresponding to the mold indicated by the mold identification information received by the reception unit;
A conversion unit that performs coordinate conversion of the analysis processed product shape data so that the analyzed processed product shape data overlaps with the imaging processed product shape data;
One piece of analysis work product shape data coordinate-converted by the conversion unit, the analysis information corresponding to point information indicating one or more points on the surface of the three-dimensional shape indicated by the analysis work product shape data, and one of the projection setting information An analysis image that is an image of a projection diagram showing information on the analysis result indicated by the analysis information on the surface of the shape indicated by the analysis processed product shape data according to the setting indicated by the projection setting information. An analysis image generation unit to perform,
A program for functioning as an output unit that outputs an image obtained by superimposing a captured image corresponding to the projection setting information used by the analysis image generation unit and the analysis image.