JP6842239B2 - Fitting simulation system and program - Google Patents

Description

The present invention relates to a fitting simulation system and a program for simulating a user who wishes to try on a fitting object on a video.

As a means of performing a fitting simulation, for example, there is a fitting simulation method as in Patent Document 1. In this fitting simulation method, the mannequin image is deformed according to the customer's person image, the face part of the person image is combined with the mannequin image, the joint is smoothed, and the clothes desired by the customer correspond to the mannequin image. After transforming it into a mannequin image, it is combined with the mannequin image.

Japanese Patent Application Laid-Open No. 9-106419

However, in the case of the fitting simulation method described in Patent Document 1, it is difficult to deform the clothes desired by the customer so as to correspond to the mannequin image. In particular, when trying-on simulation using 3D computer graphics is performed, there is a drawback that the amount of calculation becomes enormous and it is not practical.

The present invention has been conceived in view of such circumstances, and an object of the present invention is to provide a fitting simulation system and a program that can withstand practical use in performing fitting simulation by three-dimensional computer graphics.

In the following, the specific contents of the embodiment corresponding to each means for solving the problem are shown in parentheses.

The present invention is a fitting simulation system that simulates a user who wishes to try on a fitting object (for example, a suit, a jacket, a shirt, shoes, etc.) by using three-dimensional computer graphics.
A means of storing multiple types of images of the torso limbs below the head (for example, the torso limbs database 14) and
An image selection means (for example, S75 in FIG. 5B) for selecting an image suitable for the user from the body limb part images stored in the body limb part image storage means.
A generation means (for example, S76 to S80 in FIG. 5B) that joins a torso limb part image selected by the image selection means and a photographed image of the user's head to generate a user's alter ego image.
An alter ego image storage means (for example, member database 16, S58, S60, S65 in FIG. 4) for storing the alter ego image generated by the generation means, and
Specific means for identifying the user's alter ego image from the alter ego images stored in the alter ego image storage means (for example, S85, S86, S100, S101, S110, S111 in FIG. 6).
A fitting selection receiving means (for example, a display unit (touch panel) 26 of the tablet terminal 8 and S120 to S124 of FIG. 7 ) that accepts an operation of selecting an image of a plurality of types of fitting objects stored in advance to be tried on. , S128, S137, S139, S150 to S153, S160 to S163a in FIG. 8, S175 to S178, S182, S183, S190 to S192 in FIG. 9A).
Using the image of the object to be tried on selected by the operation received by the fitting selection receiving means, the fitting means on the image (for example, FIG. 7) is used to simulate fitting the alter ego image specified by the specific means on the image. S130, S165 in FIG. 8, S194 in FIG. 9 (a), and
Selection control is performed to select a fitting object image that seems to match the alter ego image stored in the alter ego image storage means as a recommendation target, and the recommendation target selected by the selection control is used as the alter ego image. With a recommendation means (for example, the flowchart of FIG. 10 or the flowchart of FIGS. 19 and 20 and the agent engine 43 for machine learning) that transmits and recommends the simulation image tried on on the video to the user corresponding to the alter ego image. , With
The recommendation means is a means for performing the selection control, and stores a plurality of types of fitting object images in association with a priority, and recommends from the plurality of types of fitting object images based on the priority. Including a fitting selection means for selecting a fitting list to be used and having a clerk select a recommendation target from the selected fitting list.
The fitting selection means weights the priority and selects the fitting list as the final priority based on the action data for promoting the consumption behavior of the user updated by the reinforcement learning means.
The reinforcement learning means receives data that can identify the consumption behavior of the user recommended by the recommendation means (for example, the purchase amount of the user / the number of recommendations) as state data for the consumption environment, and is based on the state data. The action data is updated so as to maximize the reward in the given consumption environment, and the updated action data is fed back to the consumption environment.

Further, the fitting means on the image includes a first type of fitting object image (for example, a purchased product image) and a second type of fitting object image (for example) selected by the operation received by the fitting selection receiving means. , The product image of the selected category) may be configured to be able to perform a simulation of trying on the alter ego image (for example, after storing the fitting image of the purchased product as the fitting history (S168)). The fitting history is displayed (S194), and the product images selected by selection from the product category (S195) are superimposed and displayed (S130).

Further, a purchased product storage means (for example, the member database 16 and the flowchart of FIG. 11) for storing information that can identify the purchased product that the user has purchased in the past is further provided.
The fitting selection receiving means can accept an operation of selecting from the images of the purchased products of the user specified by the storage information of the purchased product storage means (for example, the fitting history of the tried-on images of the purchased products). After memorizing as (S168), select the fitting history (S192)),
In the video fitting means, the purchased product of the user selected by the operation received by the fitting selection receiving means is used as the first type of fitting target image, and the operation received by the fitting selection receiving means. The user's unpurchased product selected by the above may be configured to be able to perform a simulation in which both the purchased product and the unpurchased product are tried on the alter ego image as the second type of fitting target image (for example). , After storing the fitting image of the purchased product as the fitting history (S168), the fitting history is displayed (S194), and the product image selected by the selection from the product category (S195) is superimposed and displayed (S130). )).

Further, the fitting selection receiving means by the reception is operated, the user terminal (e.g., a smart phone 2) carried by the user is an operation for selecting the (e.g., FIG. 17 (a) (b), FIG. 18 (a) (b )),
The fitting means on the image may display the fitting simulation on the display screen of the user terminal (for example, FIGS. 17 (a) (b) and 18 (a) (b)).

According to the present invention, the following effects are obtained. The part to be tried-on simulated is mainly the torso limb part below the head in the user alter ego image, and the torso limb part is selected from a plurality of types of torso limb images prepared in advance. , The image of the object to be tried on can be prepared according to the prepared body limb images of a plurality of types, and the fitting simulation can be easily executed.

It is the whole block diagram of the fitting simulation system. (A) is a block diagram showing a control circuit of a tablet terminal and a smartphone, and (b) is a block diagram showing a control circuit of a store server and a management server. (A) is a flowchart showing the main routine of the tablat terminal, the store server, and the management server, and (b) shows each subroutine program of the new user registration terminal processing, the new user registration store processing, and the new user registration management processing. It is a flowchart. It is a flowchart which shows each subroutine program of a user alter ego CG creation process, a user alter ego CG transmission process, and a user alter ego CG registration process. (A) is a flowchart showing a subroutine program of neck extension processing, and (b) is a flowchart showing a subroutine program of connection processing. It is a flowchart which shows each subroutine program of fitting simulation terminal processing, fitting simulation store processing, and fitting simulation management processing. It is a flowchart which shows each subroutine program of the fitting process from a product category and the fitting process corresponding to a product category. It is a flowchart which shows each subroutine program of the fitting process from the purchase history, the purchase history fitting correspondence process, and the purchase history fitting management process. (A) is a flowchart showing a subroutine program of fitting processing from the fitting history, and (b) is a flowchart showing a subroutine program of user alter ego operation processing. It is a flowchart which shows each subroutine program of the recommendation terminal processing, the recommendation store processing, and the recommendation management processing. (A) is a flowchart showing each subroutine program of purchase terminal processing, purchase store processing, and purchase management processing, and (b) is each subroutine program of purchase terminal processing, purchase store processing, and purchase management processing, and an EC management server. It is a flowchart which shows the control operation of. (A) and (b) are diagrams showing a display screen of a tablet terminal. (A) is a diagram showing a neck position specifying logic, and (b) is a diagram showing a display screen of a tablet terminal. (A) and (b) are diagrams showing a display screen of a tablet terminal. (A) and (b) are diagrams showing a display screen of a tablet terminal. It is a figure which shows the display screen of a tablet terminal. (A) and (b) are diagrams showing a display screen of a smartphone. (A) and (b) are diagrams showing a display screen of a smartphone. It is a flowchart which shows each subroutine program of the recommendation terminal processing, the recommendation store processing, and the recommendation management processing in another embodiment. (A) is a flowchart showing a subroutine program of recommendation processing based on the complex network theory, and (b) is a flowchart showing a subroutine program of reinforcement learning processing. (A) is a diagram showing the principle of the user base network, and (b) is a diagram for explaining reinforcement learning in a consumption environment.

The overall configuration of the fitting simulation system according to the present embodiment will be described with reference to FIG. Group management company of user's smartphone 2, store (chain store) 3 ... which is a retail store for clothes, etc., contractor 4, store (chain store) 3 ... which orders and tailors custom-made clothes, etc. Data centers 5 are connected to each other via the Internet 1 so that information communication is possible. Further, the EC management server 81 of the EC site 80 that performs electronic commerce is connected to the Internet 1.

A store server 6 and a wireless LAN router 7 are installed in each store 3 ..., and the tablet terminal 8 owned by the clerk 11 can communicate with the store server 6 via the wireless LAN router 7. It is configured. The store server 6 is communicably connected to the management server 13 of the data center 5 via the Internet 1 described above. The store 3 in the present embodiment is a store that sells products worn by the user, such as suits, jackets, shirts, ties, and shoes. When custom-made these products, a state in which a customer (also referred to as a user) 12 has tried on the products is displayed as an image on the tablet terminal 8 using three-dimensional computer graphics (hereinafter referred to as “CG”). The product image data to be tried on is stored in the product database 9. The head of the customer 12 is scanned in 3D by the structure sensor 99 to create a user alter ego CG, and the product image stored in the product database 9 is tried on on the tablet terminal 8 for the user alter ego CG.

A management server 13 is installed in the data center 5. The management server 13 is configured to be able to access the body limb database 14, the member database 16, and the product database 15. The body limb database 14 stores a plurality of types (for example, 5 types) of body limb CG so that the user can select a body limb CG suitable for the body shape of the user (customer). The body limb CG stored in the body limb database 14 is downloaded and stored in each tablet 8 via the Internet 1 and the store server 6. The head CG of the user 12 and the body limb CG captured by the structure sensor 99 (see FIG. 2A) are joined by the tablet terminal 8 to create a user alter ego CG. The user alter ego CG is transmitted to the management server 13 and registered in the member database 16. The user alter ego CG may also be registered in the store server 6.

In the member database 16, the store number assigned to each store 3, the member number of the registered user, the date of birth of the user, the user alter ego CG of the whole body, the purchase history data of the user purchasing the product, and the like. The e-mail address of the user's smartphone 2 is associated and registered. Further, in the product database 15, all the product images to be tried on are registered in the product database 9 of each group store 3. Of the data stored in the member database 16, the member number, date of birth, user alter ego CG, purchase history, smartphone email address, etc. corresponding to the store number of the own store 3 are stored in the store server 6 of the store 3. May also be remembered.

Further, as will be described in detail later, at each store 3, the clerk 11 pushes and distributes the coordinated image of the recommended product image on the user's alter ego CG to the smartphone 2 of the customer 12, and recommends the product. May be done. Such a recommended product history is registered in the recommendation history database 10.

Since the hardware circuits of the tablet terminal 8 and the smartphone 2 are almost the same, the hardware circuits will be described with reference to FIG. 2A. CPU (Central Processing Unit) 20 as a control center, ROM (Read Only Memory) 22 for storing control programs and control data, RAM (Random Access Memory) 21 for functioning as a work area for CPU 20, and data can be rewritten. The EEPROM (Electrically Erasable Programmable Read-Only Memory) 23 to be stored is connected by the bus 25. The bus 25 is connected to various devices via the interface unit 24. Examples of various devices include a display unit (touch panel) 26, a voice output unit 27, a voice input unit 28, a wireless communication processing unit 29, and the structure sensor 99 described above. The structure sensor 99 is provided only in the tablet terminal 8 and not in the smartphone 2. The wireless communication processing unit 29 enables the tablet terminal 8 to communicate with the store server 6 via the wireless LAN router 7, and also enables data communication with various websites and data centers 5 via the Internet 1. Further, the smartphone 2 can communicate with various websites and the data center 5 via the Internet 1 by the wireless communication processing unit 29.

Since the store server 6, the management server 13, and the EC management server 81 have substantially the same hardware configuration, the hardware configuration will be described with reference to FIG. 2 (b).

A CPU 30 as a control center, a ROM 32 that stores a control program and control data, and a RAM 31 that functions as a work area of the CPU 30 are connected by a bus 34. The bus 34 is connected to the communication unit 35, the display unit 36, and the input operation unit 37 via the interface unit 33. The communication unit 35 enables the store server 6 to perform data communication with the management server 13 of the data center 5 via the Internet 1, and the management server 13 can perform data communication with the store server 6 of each store 3 via the Internet 1. Become.

Next, the control operation of the tablet terminal 8, the store server 6, and the management server 13 for performing the fitting simulation will be described. First, a flowchart of the main program of these control operations will be described with reference to FIG. 3 (a). The CPU 20 of the tablet terminal 8 performs a new user registration terminal process in step 1 (hereinafter simply referred to as “S”) 1. If the clerk 11 performs the member registration procedure of the customer (user) 12 by operating the tablet terminal 8 with the consent of the customer 12 who has visited the store 3, the registration procedure data is transmitted to the store server 6 and S10. The new user registration store process is performed by S20, and the CPU 30 of the store server 6 transmits the registration procedure data to the management server 13 of the data center 5, so that the new user registration management process is performed by the CPU 30 of the management server 13 by S20. Will be.

Next, the above-mentioned user alter ego CG creation process of the user 12 who has completed the registration procedure is performed by S2. If the strategyd user alter ego CG data is transmitted from the tablet terminal 8 to the store server 6 via the wireless LAN router 7, the user alter ego CG transmission process is performed by S11, and the user alter ego CG data is transmitted to the management server 13. Will be sent to. Then, the management server 13 performs a process of registering the user alter ego CG in the member database 16 by S21.

If the clerk 11 operates the tablet terminal 8 to perform a fitting simulation operation on the customer 12, the fitting simulation terminal processing is executed by S3. Then, the fitting simulation store processing is performed by S12, and the data necessary for the fitting simulation is exchanged between the store server 6 and the tablet terminal 8. Further, the fitting simulation management process is executed by S22 in order to exchange data necessary for the fitting simulation between the store server 6 and the management server 13. If the clerk 11 operates the tablet terminal 8 to perform the above-mentioned product recommendation operation, the recommendation terminal processing is executed by S4, and the recommendation store processing of S13 by the store server 6 is performed to exchange the data necessary for the recommendation. Is executed. Further, the recommendation management process by S23 is executed in order to exchange data necessary for the recommendation between the store server 6 and the management server 13.

When the customer 12 purchases the product to be tried on as a result of the fitting simulation, the tablet terminal 8 executes the purchase terminal process of S5. The purchase store process by S14 is executed so that the data necessary for the purchase process can be exchanged between the tablet terminal 8 and the store server 6. Further, the purchase management process 24 is executed so as to exchange data necessary for the purchase process between the store server 6 and the management server 13.

Next, the flowchart of the subroutine program of the new user registration terminal process by S1, the new user registration store process by S10, and the new user registration management process by S20 will be described with reference to FIG. 3 (b). First, the CPU 20 of the tablet terminal 8 determines whether or not there is access to the fitting simulation of S30, and if not, the new user registration terminal process returns and shifts to the user alter ego CG creation process of S2. On the other hand, if the clerk 11 operates the tablet terminal 8 to access the fitting simulation, the determination of YES is made by S30 and the process proceeds to S31, and the user information input setting screen displayed on the tablet terminal 8 (FIG. 12 (a)). ) User search / sign-up screen and user information setting screen), enter the registered store you want to register as a member, the date of birth, and the email address of your smartphone, and send the input data to the store server 6. The user 12 may want to register not only the store 3 currently visiting, but also, for example, a store 3 that stops on the way to work, a store 3 near the office, a store 3 near the business trip destination, and the like. In such a case, you can select and register all the stores you want to register as a member.

The store server 6 receives the transmission data by S35 and transmits the reception data to the management server 13 by S36. The management server 13 receives the transmission data in S45, generates a new membership number in S46, and returns it to the store server 6. The store server 6 receives the reply data in S37, and registers the received member number and the date of birth of the user in S38. On the other hand, in the management server 13, the store number, the member number, the date of birth, and the e-mail address of the smartphone are registered in the member database 16 by S47. In the member database 16, as shown in FIG. 1, the store number, the member number, the date of birth, and the e-mail address of the smartphone are associated and stored based on the data received by S45.

Next, a flowchart of a subroutine program of the user alter ego CG creation process of S2, the user alter ego CG transmission process of S11, and the user alter ego CG registration process of S21 will be described with reference to FIG. S50 determines whether or not the camera icon 53 (see FIG. 12A) has been tapped, and if not, the user alter ego CG creation process returns. On the other hand, if the clerk 11 taps the camera icon 53 of the tablet terminal 8, YES is determined by S50, the control proceeds to S51, and the structure sensor 99 starts capturing the head of the customer (user) 12. Then, the CG of the head of the user 12 is generated and displayed by the display unit 26 of the tablet terminal 8 by S52 (see the user alter ego head creation screen of FIG. 12A). Next, the user 12 operates the display unit (touch panel) 26 of the tablet terminal 8 to set the position of the cut cross section at the neck portion (S53). The state is shown on the user alter ego head confirmation screen of FIG. 12A to be purchased. Then, the head CG is cut according to the set cut cross section. Next, the process proceeds to S54, and it is determined whether or not the cut is made directly below the chin line. When the cut is made just below the chin line, the neck portion does not exist, so the control proceeds to S55 and the neck extension process is performed. If the neck is cut instead of the chin line, S54 determines NO and proceeds to S56 without processing S55. In S56, a process of joining the head CG and the body limb portion CG is performed.

Next, the control proceeds to S57, and a process of adjusting the user alter ego CG after the connection is made is performed (see the user alter ego adjustment screen of FIG. 12B). This process is a process of adjusting the size of the head and the position of the head. Next, the control proceeds to S58, and a process of transmitting the member number, date of birth, and user alter ego CG of the user 12 to the store server 6 is performed. The store server 6 receives the transmission data by S60, and transmits the reception data to the management server 13 by S61. The management server 13 registers the user alter ego CG in the member database 16 by associating the member number with the date of birth based on the received data by S65. Next, the neck extension process of S55 and the joining process of S56 will be described with reference to FIGS. 5 and 13. First, the neck extension process will be described with reference to FIG. 5 (a). According to S70, the cut neck cross section is cut out to specify the polygon surrounding the cut out area. Next, according to S71, the polygon and the texture are extended in the direction perpendicular to the plane in which the polygon is generated and cut out.

Next, the connection process will be described with reference to FIGS. 5 (b) and 13. According to S75, a plurality of types (for example, 5 types) of body limb CGs that match the user are selected. This torso limb CG is registered in the above-mentioned torso limb database 14, but the torso limb CG is received by the store server 6 via the management server 13 and the Internet 1, and each of them is received from the store server 6. It is delivered to the tablet terminal 8 and the body limb CG is stored in the tablet terminal 8. A plurality of types of body limb CGs are displayed on the display unit (touch panel) 26 of the tablet terminal 8, and the body limbs CG matching the height and weight of the user 12 is selected.

Next, in S76, the center of gravity of the head in the head CG is obtained, the point C1 obtained by projecting the center of gravity on the neck cross section is obtained, and the four points B1, B2, B3, and B4 symmetrical with respect to the C1 are obtained. Is done. This state is shown in FIG. A point C1 obtained by projecting the center of gravity onto the neck cross section is obtained, and four points B1, B2, B3, B4 of the edge portion around the neck cross section symmetrical with respect to the point C1 are acquired.

Next, the process proceeds to S77, and the enlargement / reduction ratio of the head CG is specified from the coordinates of the four points B1, B2, B3, and B4. Next, the process proceeds to S78, and a process of specifying the center of gravity of the head to be joined is performed based on the Head Anchor object preset in the selected body limb portion.

Next, according to S79, the size of the head CG is adjusted according to the specified enlargement / reduction ratio, the position of the head CG is adjusted according to the specified center of gravity, and the head CG and the body limb CG are joined. Is done. Next, in S80, a process of adjusting the skin color of the body limbs with a color chip is performed, and this connection process returns.

Next, the flowchart of the subroutine program of the fitting simulation terminal processing of S3, the fitting simulation store processing of S12, and the fitting simulation management processing of S22 will be described with reference to FIG. It is determined whether or not the user search operation has been performed by inputting the membership number and the date of birth on the tablet terminal 8 (S85). If there is no user search operation, this fitting simulation terminal processing is returned. On the other hand, the clerk 12 opens the user search / sign-up screen on the display unit (touch panel) 26 of the tablet terminal 8 (see FIG. 13 (b)), inputs the member number and the date of birth, and taps the Search button 54. Then, the determination of YES is made by S85, the control proceeds to S86, and the input member number, date of birth, and user search request are transmitted to the store server 6. The store server 6 receives the transmission data by S100, and transmits the reception data to the management server 13 by S101. The management server 13 receives the transmission data by S110, authenticates the user by the received member number and the date of birth by S111, and stores the member number and the date of birth in association with each other. The user's alter ego CG is specified from the member database 16. Next, the process proceeds to S112, and the specified user alter ego CG is returned to the store server 6.

The store server 6 receives the reply data by S102, and transmits the received data to the tablet terminal 8 by S103. In the tablet terminal 8, the transmission data is received by S87, and the received user alter ego CG is displayed as an initial display (TOP screen) by S89 (see FIG. 13B). If the store server 6 stores the user alter ego CG, the user alter ego CG may be transmitted directly from the store server 6 to the tablet terminal 8 without communicating with the management server 13, and the user alter ego CG may be transmitted to the tablet terminal 8. If is stored, the user alter ego CG can be displayed directly on the tablet terminal 8 without communicating with the store server 6.

There are three types of fitting processing: one that is selected from the product category and tried on, one that is selected from the user's past purchase history and tried on, and one that is selected from the fitting history and tried on. Each fitting process is executed by. The fitting process from the product category is to select the product category that the user wants to try on from a plurality of product categories such as suits, jackets, shirts, and ties, select the products in that category, and try on the products. It is executed in cooperation with the product category fitting correspondence process of S104 in. The fitting process from the purchase history is to select and try on the product that the user wants to try on from the products purchased in the past, and the purchase history fitting correspondence process of S105 on the store server 6 and the purchase history fitting on S113 of the management server 13. It is executed in cooperation with the management process. In the fitting process from the fitting history, the product to be tried on again is selected from the products for which the fitting simulation has been performed on the display unit (touch panel) 26 of the tablet terminal 8 and the fitting is performed.

Next, according to S93, it is determined whether or not the user information correction operation has been performed. If not, the control proceeds to S94, and if not, the control shifts to S50. When the user information button 71 on the TOP screen displayed on the display unit (touch panel) 26 of the tablet terminal 8 is tapped, the screen transitions to the user information screen and the currently registered user information is displayed (FIG. 15 (b). ), By tapping the user information correction button 72 on the user information display screen, the determination of YES is made by S93 and the control shifts to S50. Then, the user alter ego CG creation process after S50 is executed.

Next, the subroutine program of the fitting process from the product category shown in S90 and the fitting processing corresponding to the product category shown in S104 will be described with reference to FIG. 7. It is determined by S120 whether or not there is a selection from the product category, and if not, the fitting process from this product category is returned. If the product category icons 55a, 55b, 55c, 55d, 55e, 55f, 55g (see FIG. 14A) displayed on the TOP screen of the display unit (touch panel) 26 of the tablet terminal 8 are tapped, the S120 will be used. A YES determination is made and control proceeds to S121. The product category icons include, for example, an icon 55a for selecting a suit category, an icon 55b for selecting a jacket category, an icon 55c for selecting a gilet category, an icon 55d for selecting a shirt category, and a pants (trousers) category. An icon 55e for selecting, an icon 55f for selecting a tie category, and an icon 55g for selecting a shoe category are prepared. If the icon of the category desired by the customer 12 is tapped from these categories, the selected product category is transmitted to the store server 6 by S121. The store server 6 receives the transmitted data by S150, and returns the data of the product list belonging to the transmitted product category to the tablet terminal 8 by S151. In the tablet terminal 8, the returned product list data is received by S122, and the product list is displayed by the display unit (touch panel) 26 by S123 (see the TOP screen of FIG. 14A).

If the store server 6 stores the product list data belonging to the product category, the product list data may be directly transmitted from the store server 6 to the tablet terminal 8 without communicating with the management server 13, and the product. If the tablet terminal 8 stores the list data, the product list data can be displayed directly on the tablet terminal 8 without communicating with the store server 6.

Next, S124 determines whether or not the product to be tried on is selected from the listed products, and if not selected, S125 determines whether or not the product selection is completed, and if not. It is determined by S137 whether or not there is a search operation, and if not, the control returns to S124.

If a product to be tried on is selected from the products listed by S123 during the circulation of the loop of S124 → S125 → S137 → S124, a YES judgment is made by S124 and the control proceeds to S128. The process of transmitting the selected product number to the store server 8 is performed. The store server 8 receives the transmission data by S150, and returns the product image corresponding to the received product number to the tablet terminal 8 by S153. In the tablet terminal 8, the reply data is received by S129, and the received product image is tried on and displayed on the user alter ego CG by S130. If the store server 6 stores the product image data, the product image data may be transmitted directly from the store server 6 to the tablet terminal 8 without communicating with the management server 13, and the product image data may be transmitted to the tablet terminal 8. If the data is stored, the product image data can be displayed directly on the tablet terminal 8 without communicating with the store server 6.

Next, the user alter ego operation process is performed by S131. This user alter ego operation process is a process of rotating, enlarging or reducing the user alter ego CG in the fitting state, or modifying a part of the product in the fitting state, which will be described later based on FIG. 9B. ..

Next, it is determined by S132 whether or not there is a history registration operation. If not, the process proceeds to S134, but if the fitting history button 60 (see FIG. 14B) is tapped, a YES judgment is made from S132, the control proceeds to S133, and the current fitting display state is stored in the clipboard. Is done.

Next, it is determined by S134 whether or not the user purchases the product being tried on, and if it is not purchased, the control proceeds to S136, but if it is purchased, the purchase registration is made by S135. This purchase registration is a process for registering a product image in a fitting state as a purchased product and placing an order for the registered product as a made-to-order product to the contractor 4. Next, it is determined by S136 whether or not there is a reset operation, and if not, the control shifts to S126. If the reset icon 58 (see FIG. 14A) is tapped, S136 determines YES and the control shifts to S89.

On the other hand, if the search icon 59 (see FIG. 14A) is tapped after the product list is displayed by S123, a YES judgment is made by S137, the control proceeds to S138, and the listed products are displayed. Products that match the search conditions are selected from the list. Then, according to S139, the details of the selected product are displayed (see the search condition display screen of FIG. 14A). Then, control shifts to S128.

When the product selection is completed, YES is determined by S125, the control proceeds to S126, it is determined whether or not the fitting from the product category is completed, and if it is not completed yet, S127 is used to determine the product category. It is determined whether or not there has been a selection (including a selection of a category different from the already selected category). If not, control returns to S126. If any of the product category icons 55a to 55g is tapped during the cycle of the loop of S126 → S127 → S126, S127 determines YES and the control shifts to S121. On the other hand, when the fitting end operation from the product category is performed, a YES determination is made by S126, and the fitting process from this product category is returned.

Next, a flowchart of a subroutine program of the fitting process from the purchase history of S91, the purchase history fitting correspondence process of S105, and the purchase history fitting management process of S113 will be described with reference to FIG. It is determined by S160 whether or not there is a purchase history display operation, and if not, the fitting process from the purchase history is returned. If the purchase history button 63 (see FIG. 15A) displayed on the display unit (touch panel) 26 of the tablet terminal 8 is tapped, YES is determined by S160, the control proceeds to S161, and the purchase history request is made. It is transmitted to the store server 6. The store server 6 receives the transmission data by S175, and transmits the reception data to the management server 13 by S176. The management server 13 receives the transmitted data from S182, searches the member database 16 for a list of purchase histories of the user and image data of purchased products by S183, and returns the data to the store server 6. When a user purchases a product at a plurality of registered stores, the purchase history data at all the stores is stored in the member database 16, and the list of all the purchase histories and the image data of the purchased product are stored in the store server. Reply to 6.

The store server receives the returned data by S177, and returns the received data to the tablet terminal 8 by S178. In the tablet terminal 8, the reply data is received by S162, and the received purchase history list is displayed as a purchase history list screen by S163 (see FIG. 15A). When the store server 6 stores the list of purchase history and the image data of the purchased products, the data may be transmitted directly from the store server 6 to the tablet terminal 8 without communicating with the management server 13. Further, if the tablet terminal 8 stores the data, the data can be displayed directly on the tablet terminal 8 without communicating with the store server 6.

Next, it is determined whether or not the product to be tried on is selected from the products listed by S163, and if it is not selected, it is determined by S164 whether or not the fitting operation from the purchase history has been completed, and the end operation is performed. If not, control returns to S163.

In the middle of the loop of S163 → S163a → S164 → S163, if the user 12 selects the purchased product to be tried on from the purchased products displayed in the purchase history list and taps the corresponding user alter ego display button 61, S163a The determination of YES is made and the control proceeds to S165, and the selected product image is controlled to be tried on and displayed on the user alter ego CG (see the TOP screen of FIG. 15A). Next, the user alter ego operation process is performed by S166, and then it is determined by S167 whether or not the history registration operation has been performed. This history registration operation is an operation of registering the product currently being tried on as a fitting history, and if there is no history registration operation, the process proceeds to S169. On the other hand, if the clip icon 56 is tapped, a YES determination is made by S167, the control proceeds to S168, and a process of storing the current fitting display state in the clipboard is performed. Then, it is determined by S169 whether or not there is a reset operation, and if there is no reset operation, the control shifts to S163a, but if there is, the control shifts to S89. Further, if the operation of ending the fitting from the purchase history is performed, a YES determination is made by S164, and the fitting process from the purchase history is returned.

Next, the flowchart of the subroutine program of the fitting process from the fitting history shown in S92 will be described with reference to FIG. 9A. S190 determines whether or not there has been a fitting history display operation, and if not, the fitting process from this fitting history returns. If the fitting history button 60 (see FIG. 14B) displayed on the TOP screen of the display unit (touch panel) 26 of the tablet terminal 8 is tapped, YES is determined by S190, the control proceeds to S191, and fitting is performed. The history list screen is displayed (see FIG. 14B). Next, S192 determines whether or not the fitting history selection operation has been performed. If not, S193 determines whether or not the operation to end the fitting from the fitting history has been performed. If not, the control is S191. Return to. During the cycle of the loop of S191 → S192 → S193 → S191, the user 12 selects a product to be displayed from a plurality of fitting history products displayed on the fitting history list screen, and the corresponding user alter ego display button 61. If is tapped, a YES determination is made by S192, the process proceeds to S194, and control is performed to display the selected fitting history on the display unit (touch panel) 26. (See FIG. 14 (b)). As a result, the tried-on product fitting image already tried on by the tablet terminal 8 and stored in the clipboard, and the purchased product purchased by the user 12 and registered as purchase history data in the member database 16 are described above. The fitting state product image stored in the clipboard by S168 of the above is displayed by the display unit (touch panel) 26.

Next, according to S195, it is determined whether or not there has been a product category selection operation. The customer 12 may want to try on a new product image by superimposing a new product image on the purchased product image or the tried-on product image. For example, you may want to try on a new tie over a purchased suit or a suit you have tried on and determine if the tie matches those suits. In such a case, the icon of the category to be tried on and displayed is tapped from the product category icons 55a to 55g. If such a product category icon is tapped, a YES determination is made by S195, control shifts to S121, fitting processing from the product category is performed, and overlapping fitting is possible. On the other hand, if there is no selection operation from the product category, the control proceeds to S196, the user alter ego operation process is performed, it is determined by S197 whether or not there is a history registration operation, and if not, the process proceeds to S199. In the case, S198 performs a process of storing the current fitting display state in the clipboard.

Next, it is determined whether or not to purchase the fitting product displayed by S199, and if it is purchased, the purchase is registered by S200, and then the control proceeds to S201. In S201, it is determined whether or not there is a reset operation, and if there is no reset operation, the control shifts to S192, but if there is, the control shifts to S89. On the other hand, if the operation of ending the fitting from the fitting history is performed, a YES determination is made by S193, and the fitting process from the fitting history is returned.

Next, the flowchart of the subroutine program of the user alter ego operation processing shown in S131, S166 and S196 will be described with reference to FIG. 9B. S210 determines whether or not there is a swap operation, if not, S212 determines whether or not there is a pitch-in operation, if not, S214 determines whether or not there is a pitch-out operation, and if not, S216 determines whether or not there is a pitch-out operation. It is determined whether or not there is an attachment / detachment operation, and if there is no operation, it is determined by S218 whether or not there is a partial correction operation of the product. If there is no operation, this user alter ego operation process returns.

If a swap operation is performed on the fitting state image (see, for example, FIG. 14A), the control proceeds to S211 and the displayed user alter ego is rotated and displayed. If the pitch-in operation is performed on the fitting state image, the control proceeds to S213, and the displayed user alter ego is reduced and displayed. If the pitch-out operation is performed on the fitting display state image, the control proceeds to S215, and the control to enlarge and display the user alter ego CG is performed.

When the product attachment / detachment icon 57 is tapped, the control proceeds to S217, and the product attachment / detachment display of the product being tried on is displayed. If the product attachment / detachment icon 57 is tapped in the fitting state, the product is in the erased display state, and if the product attachment / detachment icon 57 is tapped again in the erased state, the erased product is tried on again. It will be displayed.

When a partial correction operation of the product is performed, the control proceeds to S219, and display control for correcting a part of the displayed fitting product is performed. The partial modification display control of this product, for example, modifies a three-button suit to a two-button suit, or unbuttons the suit image with the button closed so that the gilet worn under it can be seen. You can modify it to the image you made, or modify the two-tuck trousers to one-tuck trousers. Based on the partially modified display control of this product, not only the existing product image registered in the product database 9 but also the product image customized for the user by partially modifying it to reflect the user's tastes and wishes. Custom-made is possible.

Next, a flowchart of a subroutine program of the recommendation terminal process of S4, the recommendation store process of S13, and the recommendation management process of S23 will be described with reference to FIG. It is determined by the tablet terminal 8 whether or not there is a recommendation operation (S225), and if not, the recommendation terminal process returns. On the other hand, there is a case where the clerk 11 creates a fitting simulation image coordinated with the recommended product for the corresponding user 12, introduces it to the user, and recommends it. In such a case, the clerk 11 inputs the member number and date of birth of the member to be recommended in the Coordinate field of the user search / sign-up screen and taps the Search button 54 (see FIG. 16). Then, a YES determination is made by S225 and a YES determination is made by S226, and the control proceeds to S227. In S227, the entered member number, date of birth, and recommendation history request are transmitted to the store server 6. The store server 6 receives the transmission data by S240, identifies the user by S241, searches the recommendation history list data from the recommendation history database 15, and returns the searched recommendation history list data to the tablet terminal 8 by S242. To do.

In the tablet terminal 8, the reply data is received by S228, and the recommendation history list is displayed by S229 (see FIG. 16). The clerk 11 refers to the list of products recommended in the past, and selects the product newly recommended this time by using the fitting simulation terminal processing of S230, the fitting simulation store processing of S243, and the fitting simulation management processing of S255. The fitting simulation terminal processing S230 has the same control as the fitting simulation terminal processing of S3, the fitting simulation store processing of S243 has the same control as the fitting simulation store processing of S12, and the fitting simulation management processing of S255 is the fitting simulation of S22. It is the same control as the simulation management process. If the tablet terminal 8 stores the recommendation history, the recommendation history list can be displayed on the tablet terminal 8 without communicating with the store server 6. Further, the recommendation history made in all the chain stores 3 is stored in the member database 16, and the total history list of the recommendations made by all the chain stores 3 for a certain user 12 is stored from the management server 13 via the store server 6. It may be possible to send to the tablet terminal 8 so that the total recommendation history list can be displayed.

By these processes, the clerk 11 taps the clip icon 56 for the product to be recommended and registers the recommendation (see FIG. 16). Next, S231 determines whether or not there is a fitting history that you want to recommend, and if there is no try-on history, it returns, but if there is a recommended registered product, S231 makes a YES judgment, control proceeds to S232, and the user's email. Control is performed to send the address request to the store server 6. This is done by tapping the send button 64 (see FIG. 16) on the recommendation registration screen.

The store server 6 receives the user's e-mail address request by S244, and transmits the received data to the management server 13 by S245. The management server 13 receives the transmission data by S256, searches the user's e-mail address from the member database 16 by S257, and returns it to the store server 6. The store server 6 receives the reply data by S246, and returns the received data to the tablet terminal 8 by S247. The tablet terminal 8 receives the returned data by S233, and controls to transmit the recommendation fitting CG to the received e-mail address by S234. As a result, the recommendation fitting CG is push-delivered to the user's smartphone 2. Then, the new recommendation history data is transmitted to the store server 6 by S235, the transmission data is received by S248 in the store server 6, and the received recommendation history is additionally registered in the recommendation history database 10 by S249. If the store server 6 stores the user's e-mail address, push distribution can be performed to the e-mail address without communicating with the management server 13.

Next, a flowchart of a subroutine program of the purchase terminal process of S5, the purchase store process of S14, and the purchase management process of S24 will be described with reference to FIG. First, referring to FIG. 11A, the tablet terminal 8 determines whether or not there is a purchase registration by S260. If there is no purchase registration, this purchase terminal processing returns. On the other hand, when the purchase is registered by S135 and S200, YES is determined by S260, the process proceeds to S261, and the registered product image at the time of fitting is transmitted to the store server 6 as order data. The store server 6 receives the transmission data in S265, transmits the product image of the received data to the contractor 4 in S266, and performs the order processing. Next, the order data is transmitted to the management server 13 by S267.

The management server 13 receives the transmission data by S270, and performs a process of registering the image data of the purchased product in the member database 16 as the purchase history of the user by S271.

FIG. 11A shows a case where the store 3 places an order with the contractor 4, while FIG. 11B shows a case where the EC site 80 makes a payment and places an order with the contractor 4. Here, the differences from FIG. 11A will be mainly described. The store server 6 accesses the EC management server 81 after receiving the data from the tablet terminal 8 by S265 (S345). The management server 81 that has received the access (S355) returns a payment page by S356. The store server 6 that has received it in S346 transmits the received data to the tablet terminal 8 in S347. The tablet terminal 8 that has received it in S340 displays the payment page on the display unit (touch panel) 26 (S341). When the user (customer) 12 who sees the data inputs the items necessary for payment, the input data and the product image of the order data are transmitted to the store server 6 (S342). The store server 6 receives it (S348) and transmits the received data to the EC management server 81 (S349). The EC management server 81 receives it (S357) and performs payment processing (S358). After that, the EC management server 81 transmits the product image received in S357 to the contractor 4 and performs the order processing (S359).

When making a payment and placing an order at the EC site 80 shown in FIG. 11B, it is useful when performing a fitting simulation using a smartphone 2 described later. In the case of the fitting simulation using the smartphone 2, since the user 12 performs the fitting simulation without going to the store 3, the settlement and the order cannot be made at the store 3, and in that case, the settlement and the order are placed at the EC site 80. I do.

Next, a display screen displayed on the display unit (touch panel) 26 of the tablet terminal 8 will be described with the control of the flowchart described above. First, with reference to FIG. 12A, the member registration screen of the user (customer) 12 and the registration procedure of the user alter ego CG of the user will be described.

When the clerk 11 operates the display unit (touch panel) 26 of the tablet terminal 8 to access the fitting simulation, the user search / sign-up screen is displayed. In the case of a user who is about to register as a member, enter the date of birth of the user in the "User Search" field, and then tap the Sign up button 50 to move to the user information setting screen. Then, when the user wants to be registered in a plurality of stores, the area can be narrowed down to the stores in the selected area by selecting an area from the pull-down menu of "registered stores" on the user information setting screen. The input information is transmitted to the store server 6 with the registered store specified (see FIG. 3B).

Note that 51 in the figure is a Back button, and by clicking this button, the user search / sign-up screen displayed immediately before is displayed. In addition, 52 in the figure is a Next button, and clicking this button transitions to the next page. To create a user alter ego CG, click the Next button 52 on the user information setting screen to display the user alter ego head creation screen, and then tap the camera icon 53. Then, the head of the user 12 is captured by the structure sensor 99 provided in the tablet terminal 8 to create a head CG (see S50 to S52). Then, in order to cut the head in the displayed head CG, the user sets the cut portion by operating the cut cross section on the user alter ego head confirmation screen (see S53). Then, by tapping the Next button 52, the screen transitions to the user alter ego adjustment screen shown in FIG. 12 (b).

This avatar adjustment screen is a specific screen of the user alter ego adjustment process by S57 after the head CG and the body limb portion CG are connected by the connection process shown in S56 of FIG. On this user alter ego adjustment screen, the suit is displayed in white for later fitting. The size of the head is adjusted by pitching in and out of this screen, and the position of the head is adjusted by two-point touch movement. The skin color of the body is adjusted with a color chip. This color chip is selected from three patterns of basic skin color, black and white. When the adjustment of the user alter ego is completed, the Next button 52 is tapped to move to the user information registration screen, and after confirming the registration information, the user alter ego CG is transmitted to the store server 6 (see S58).

FIG. 13A shows the neck position specifying logic and illustrates the process of S76 in FIG. 5 described above. C1 in FIG. 13A is the “point CG in which the center of gravity of the head is obtained and the center of gravity is projected onto the neck cross section” shown in S76. The four points symmetrical with respect to this point C1 are B1, B2, B3, and B4 in FIG. 13A.

FIG. 13B shows a user search / sign-up screen required for a user to identify the user and search for a user alter ego CG when performing a fitting simulation. By entering the membership number, date of birth, etc. in the User Reference field on this screen and tapping the Search button 54, the entered data is transmitted to the store server 6 (see S86 in FIG. 6).

Then, the data of the user alter ego CG registered in the member database 16 corresponding to the input member number and date of birth is returned to the tablet terminal 8 (see S87 in FIG. 6). Then, the received user alter ego CG is initially displayed as a TOP screen (see S89 in FIG. 6).

As a method of trying on the user alter ego CG displayed on the TOP screen, as described above, there are fitting from the product category (including fitting by searching), fitting from the purchase history, and fitting from the fitting history. As shown in FIG. 14A, product category icons 55a, 55b, 55c, 55d, 55e, 55f, 55g are displayed in the lower portion of the TOP screen. To try on from a product category, tap the icon of the category you want to try on from the product category icons 55a to 55g. FIG. 14A shows a state in which 55a, which is a suit product category icon, is tapped. In addition to this suit icon, the product category icon includes jacket category icon 55b, gilet category icon 55c, shirt category icon 55d, pants (trousers) category icon 55e, tie category icon 55f, and shoe category icon. 55g is prepared.

By selecting and tapping the icon of the category you want to try on from these product category icons, a list of images of products belonging to that category is displayed on the right side of the TOP screen (see S120 to S123). By tapping the product image that the user wants to try on from the product images displayed in the list, the state in which the product image has been tried on is displayed on the user alter ego CG (see S124 to S130).

To find the item you want to try on by searching from the product images listed by specifying the category, tap the search icon 59 on the TOP screen. Then, the screen transitions to the search condition display screen, and by inputting and specifying the search condition, detailed information and an image of the product matching the condition are displayed (see S137 to S139). By tapping the detailed display and image of the product, the product is tried on to the user alter ego CG.

The user alter ego CG in the fitting state can be rotated and displayed by swapping (see S210 and S211), reduced by the pitch-in operation (see S212 and S213), and the user alter ego CG by performing the pitch-out operation. Is enlarged and displayed (see S214 and S215). Further, by tapping the product attachment / detachment icon 57, it is possible to display the attachment / detachment of the product being tried on. By tapping the clip icon 56, the currently displayed image of the fitting state of the user alter ego CG is stored in the clipboard (see S132 and S133).

Next, fitting from the fitting history will be described with reference to FIG. 14 (b). By tapping the fitting history button 60 on the top screen, a list of fitting history recorded on the clipboard is displayed (see S190 and S191). By viewing this try-on history list and tapping the user alter ego display button 61 of the try-on history that the user wants to try on, the try-on history image is displayed on the top screen (see S192 and S194). On the other hand, the fitting history data is deleted by tapping the delete button 62 on the fitting history list screen.

Next, fitting from the purchase history will be described with reference to FIG. 15 (a). By tapping the purchase history button 63 displayed on the top screen, the screen transitions to the purchase history list screen (see S160 to S163). By tapping the user alter ego display button 61 of the purchase history that the user wants to try on from the purchase history displayed in the purchase history list, the state of trying on the product image of the purchase history is displayed (S163a, S165). reference).

For example, when the user tries on a tie that suits a suit purchased in the past, select the suit from the above-mentioned purchase history list and try it on, and in that state, tap the clip icon 56 to clipboard as the try-on history. To memorize. Then, the fitting history button 60 is tapped to display a list of fitting images of the suits purchased in the past, and the fitting image is displayed by tapping the fitting image. In that state, by tapping the necktie category icon 55f in the product category icon, the image list of the necktie is displayed. Tap the tie that suits the suit from among them and put it on the suit to try on the tie. As a result, you can choose by trying on a tie that suits the suit you purchased in the past and changing it. By tapping the product category icon 55f in the state of the fitting screen of the purchase history image without temporarily storing the fitting display image from the purchase history as a history image in the clipboard, the image of the tie is displayed in a list. By tapping the tie from the inside, you may control so that you can try on the tie on top of the suit you purchased in the past.

Next, on the top screen shown in FIG. 15B, by tapping the user information button, the screen transitions to the user information screen, and the current user information is displayed. By tapping the user information correction button 72 on the user information screen, it is possible to transition to the user alter ego head creation screen of FIG. 12A and correct the user information (see S93 and S50). Next, the display screen in the case of performing the recommendation described with reference to FIG. 10 will be described with reference to FIG. Enter the member number and date of birth of the user in the Coordinate column on the user search / sign-up screen to search for the user's alter ego CG of the user you want to recommend, and tap the Search button 54. To do. Then, the user alter ego CG of the user is searched and displayed on the top screen. By tapping the hamburger icon 65 in this state, the screen transitions to the recommendation history list screen of the user (see S229). The clerk 11 looks at this past recommendation history list and coordinates the recommendation fitting image of this time so as not to be duplicated. The coordination coordinates the fitting image that matches the user by making full use of the fitting such as the fitting from the product category and the fitting from the purchase history described above. Then, by tapping the clip icon 56, the coordinated fitting image is registered as a recommendation, and the recommended fitting image is push-delivered to the user's smartphone 2 (see S234).

17 and 18 show the display screen of the display unit (touch panel) 26 on the user's smartphone. The display unit (touch panel) 26 of the smartphone 2 also displays a fitting simulation for the user alter ego CG as in the tablet terminal 8. The control has the same contents as the control shown in FIGS. 13 to 11 described above in which the "tablet terminal" is rewritten as a "smartphone" and the control of the store server is omitted. Therefore, the control flowchart of the fitting simulation by the smartphone omits the repetition of the description. If the user 12 operates the smartphone 2 to input the membership number and the date of birth and sends it to the management server 13 (see S86), the user is authenticated by the management server 13 and the specified user alter ego CG is the smartphone 2. Is returned to (see S111 and S112). The smartphone 2 that has received it displays the received user alter ego CG on the display unit (touch panel) 26 as shown in FIG. 17 (a). In this state, by tapping the icon of the category you want to try on in the product category icons 55a, 55b, 55c, 55d, 55e displayed at the bottom of the display screen, the product list image display of the selected category is displayed at the bottom of the display screen. It is displayed in the part (see S121 to S123). By tapping the product image that the user who sees the product list display wants to try on, the image of trying on the product on the user's alter ego CG of the user is displayed (see S124 and S128 to S130).

When the search condition is input and specified while the list of product images is displayed, the product images matching the search conditions and the detailed information thereof are displayed from the list display products (see S137 to S139). Further, the user information is displayed on the screen by tapping the user information display icon 68.

With reference to FIG. 17B, tapping the product spec icon 66 transitions to the product spec screen. On this product spec screen, the image of the product, the brand name, the category, the wearing size, etc. are displayed. The price is also displayed. Considering that the screen of the smartphone is small, the product image may be controlled so as not to be displayed.

Swap and pitch in this fitting image as described above. Rotation and enlargement / reduction can be performed by performing operations such as pitch out (S210 to S215). Further, as described above, it is also possible to modify a part of the product image being tried on (see S218 and S219). Further, by tapping the product attachment / detachment icon 57, the product being tried on can be attached / detached (see S216 and S217). By tapping the clip icon 56, the fitting image currently displayed is stored in the clipboard (see S132 and S133). Further, by tapping the reset icon 58, the user alter ego CG is returned to the initial state at the time of displaying the screen (see S136 and S89). By tapping the product spec icon 66, the screen transitions to the product spec screen shown in FIG. 17 (b). By tapping the fitting history list icon 67, the screen transitions to the fitting history list screen shown in FIG. 18A.

With reference to FIG. 18A, the fitting history list screen is displayed by tapping the fitting history icon 67. On this fitting history list screen, purchase history product images stored in the member database 16 are displayed in a list. By tapping the image to be tried on from the images displayed in the list, the fitting image is enlarged and displayed (see S163a and S165). As described above, if the recommendation fitting CG coordinated by the clerk 11 is push-delivered (see S234), it is displayed by the recommended icon 69 in the fitting history list on the user's smartphone 2. The state is shown in FIG. 18 (b). By tapping the recommendation fitting CG with this recommended icon, the recommendation fitting CG is enlarged and displayed in the center of the screen. A message from the clerk 11 is displayed in the message field 70 below the recommendation fitting CG.

Next, another embodiment will be described with reference to FIGS. 19 to 21. In the above-described embodiment, the clerk 11 creates a recommendation fitting CG that matches the user 12 and push-distributes it. However, in this other embodiment, the recommendation processing and artificial intelligence based on the complex network theory are used. Reinforcement learning is performed and recommendations are made based on the learning results.

First, with reference to FIG. 19, the control of the modification of FIG. 10 described above will be described. In the management server 13, recommendation processing based on the complex network theory is performed by S305, and then reinforcement learning processing is performed by S306. The recommendation process based on this complex network theory will be described with reference to FIG. 20 (a).

It is determined by S320 whether or not it is time to update the virtual community. If it is not yet time to update, the recommendation process based on this complex network theory will return. When the virtual community update time has arrived, S320 determines YES and control proceeds to S321. This virtual community is each divided community in which users who are highly related in consumption form are grouped and divided in the user set, and the details will be described later. It is necessary to update the virtual community on a regular basis due to an increase in the number of member users, an increase in the number of recommendations to the user, and the reaction of the user as a result of the recommendation (viewing the recommendation fitting CG and purchasing products based on the recommendation). Whether or not the predetermined update time has come is determined by this S320.

In S321, a process of creating a user-based network K based on purchase history data is performed. The principle of this user base network K will be described with reference to FIG. 21 (a). In FIG. 21 (a), the number of users is narrowed down to five, A, B, C, D, and E, and the number of products is narrowed down to four products of 1, 2, 3, and 4 in order to prioritize comprehensibility. It is simplified. An arrow indicates which product in the product set each user of the user set 40 has purchased. For example, user A has purchased products 1, 2 and 3. User B has purchased products 1 and 2. The user base network 42 is a network in which users who have purchased the same product are connected by a link (also referred to as an edge). S321 creates a user base network K based on purchase history data. Therefore, this user base network K is a network in which users who have purchased the same product are connected by a link.

Next, S322 performs a process of creating a user base network S based on the fitting history data. As a result, this user base network S becomes a network in which users who have tried on the same product are connected by a link. Next, according to S323, a process of creating a user network Z based on the user attribute data input by the clerk is performed. The user attribute data input by the clerk is, for example, user preference information such as a favorite brand and a favorite color for each product category, a user's age, and the like. By responding to the customer 12, the clerk 11 detects various preference information and a rough age of the customer 12, inputs the user attribute data from the tablet 8, and sends the user attribute data to the management server 13, and the member number of the member database. And register according to the date of birth. Therefore, this user base network Z is a network in which users having the same attributes such as various preference information and age are connected by a link.

In addition to the above virtual community divisions K, S, and Z, a user base network may be created based on three data of purchase history, fitting history, and user attributes. This user-based network S is a user-based network created as the product set shown in FIG. 21 (a) based on all the sets of the purchase history product set, the fitting history product set, and the user attribute set. ..

Next, according to S345, the virtual community division K (k1, k2, k3 ...), S (s1, s2, s3 ...), Z (z1, z2, z3 ...) Based on each of the user base networks K, S, Z ... ) Is created. This virtual community division will be briefly described below. In the user base network 42 of FIG. 21 (a), each user A, B, C, D, E is used as a node, a line connecting each user is used as a link, and the relationship between the two is viewed from the viewpoint of power distribution. Follow the rules. The phenomenon of following this power law is called scale-freeness in complex network theory, and indicates that there are some self-similar behavior patterns in a wide variety of consumption activities.

In the user base network 42 having such a property, the mediation centrality of the edge is calculated. This is an application of how central the index is in the network to the link. For example, if the connection (link) between a certain user and another user is excluded, the score (index) becomes higher as the connection (link) between the communities disappears or becomes farther. The community is divided by repeating the process of removing the link having the maximum value of mediation centrality calculated for each link and dividing the network. The modularity Q index is used to determine the number of steps to divide the community (number of steps). This modularity Q is an index comparing the degree of connection within the divided community and the degree of connection between the communities. The division process is performed up to the number of division steps that maximizes the modularity Q.

In this way, virtual community division is performed for each of the user base networks K, S, and Z. K (k1, k2, k3 ...) In S325 indicates that the virtual community division was performed for the user base network K, and each virtual community division of k1, k2, k3 ... Was performed. S (s1, s2, s3 ...) And Z (z1, z2, z3 ...) have the same meaning.

Then, according to S326, a process of creating a product-based network is performed in each of the three virtual community divisions K, S, and Z. This product-based network is a network in which products purchased (or tried on) by the same user are linked to each other. However, the network for the above-mentioned user attributes is not created.

Next, according to S327, a process of prioritizing and storing products in descending order of the number of links in the product-based network for each divided community is performed. The divided community of each user base network is a divided community of users who are highly related to product purchase, a community of users who are highly related to fitting products, a community of users who are highly related to user attributes, and so on. In the community of users who are highly related to the consumption form, the products with high relevance are connected by links, and the products with a large number of links are the products with high relevance in each community. .. Therefore, when making recommendations, it is efficient to make recommendations in order from the products with the highest relevance. For example, when the user A belongs to the community division of k1, s2, and z3 of each virtual community division, the user A recommends the products with the most links in k1 in order of priority, and in s2. It is efficient to recommend products with a large number of links and high priority, and to recommend products with a large number of links and high priority within z3.

Next, the reinforcement learning process shown in S306 will be described with reference to FIGS. 20 (b) and 21 (b). First, referring to FIG. 21B, the reinforcement learning agent engine 43 is installed in the management server 13, and information is exchanged with the consumption environment S. The consumption environment S can be modeled by a set of discrete states S = {s | s ∈ S}. When the reinforcement learning agent engine 43 performs the action a on such an environment S, a reward r for it is obtained. The opportunity learning agent engine 43 acquires the data of the state s of the consumption environment S, determines the action a based on the state s, and executes the action a for the consumption environment S.

The management server 13 can exchange information with the learning database 44. In the learning database 44, state data s (purchase amount / recommendation count) and action data a (weighting) are provided for each of the above-mentioned virtual community divisions k1, k2 ..., S1, s2, ..., Z1, z2, ... Is associated and memorized. The state data s is a quotient obtained by dividing the purchase amount when the user purchases the product by the number of recommendations as a result of recommending the product to the user belonging to each virtual community division. For example, if the user purchases a suit of 13300 yen as a result of making recommendations 10 times to the user of the virtual community division k1, 13300/10 = 1330.

The action data a is a weighting value for each divided community with respect to the priority for each divided community calculated in S327 described above. For example, for the priority of the divided community k1, the weight is 7, and the quotient obtained by dividing the priority by the weight 7 is used as the final priority. Further, for example, in the case of the virtual community s2, the weighting is 5, and the quotient obtained by dividing the priority of the virtual community division S2 by 5 is used as the final priority.

Therefore, for example, in the virtual community division k1, the final priority is 1/7, 2/7, 3/7 ..., while the priority of the virtual community s2 is 1/5, 2/5, 3/5 ... Become. Then, for example, when the user is a person who belongs to both the virtual community division k1 and s2, the recommended product list is created in order from the product with the smallest priority value as the priority to recommend to the user. And send it to the tablet 8. In the above example, the order of priority is 1/7, 1/5, 2/7, 2/5, 3/7, 3/5 ....

The flowchart of the subroutine program of the reinforcement learning process of S306 will be described with reference to FIG. 20 (b). First, it is determined by S330 whether or not the state data s is received. If it is not received, this reinforcement learning process returns, but if it is received, the control proceeds to S331, and a process of updating the current state data s based on the received state data s is performed. For example, when the user of the received state data s (purchase amount / recommendation count) belongs to the virtual community division k1, the average of the state data s including the newly sent state data s is calculated. And update. For example, assuming that the number of users belonging to the virtual community division k1 is 100, since the state data s before updating is 1330, the average is calculated by (1330 × 99 + newly received state data s) / 100. .. The state data s is updated to the calculated average.

Next, the reward r is calculated by S332. The reception of the state data s by S330 is the data transmitted from the consumption environment S as a result of feeding back the previous action data a to the consumption environment S, and the reward r is based on the state data s which is the result data. It is calculated. For example, if the value of the new state data s transmitted this time is higher than the value of the state data s stored in the learning database 44 immediately before receiving the state data s, a high reward r is calculated. On the contrary, if the value is low, the low reward r is calculated.

Next, the process proceeds to S333, and the process of calculating the act a having the highest Q value as Q (a) is performed. If the value of the act a is defined as Q (a) and the correct Q (a) value (hereinafter referred to as the Q value) is obtained by the learning process, the act of maximizing the Q value is the result of learning. At first, since it is not known how much reward is obtained by performing the act a, the value of Q (a) is initialized to 0 for all the acts a. Then, the possible a go in order, to get the reward r _a at that time. Then, for each a, Q (a) = ra, and the _a with the highest Q value is obtained. The action data a stored in the learning database 44 is updated with the new action data a calculated by S333 (S334), and the action data is updated in the consumption environment S (weighting for the virtual community division to be updated). Give feedback to.

Returning to FIG. 19, it is determined whether or not the recommendation operation has been performed on the tablet 8 (S285). If there is no operation, this recommendation terminal processing returns. When the clerk 11 performs the recommendation operation, the control proceeds to S286, and the user's membership number, date of birth, and recommendation request are transmitted to the store server 6. The store server 6 receives the transmission data by S295, and transmits the reception data to the management server 13 by S296. The management server 13 receives the transmission data by S307, control proceeds to S308, authenticates the user based on the received data, and performs a process of searching the product priority for each virtual community division to which the user belongs. This priority is the priority stored by S327. Next, in S309, the quotient obtained by dividing each searched product priority by the current action data a (weighting for each virtual community division) is calculated. Next, S310 creates a recommendation list in ascending order of the products whose quotients are small, and S311 performs a process of referring to the recommendation history of the user and deleting the already recommended products from the recommendation list. Next, according to S312, the recommended product list and the e-mail address are returned to the store server 6.

The store server 6 receives the reply data by S297, and returns the received data to the tablet 8 by S298. On the tablet 8, the transmission data is received by S287, and the received recommended product list is displayed by S288. The clerk 11 who sees the list selects the product to be recommended from the list (S289). Next, according to S290, the recommendation fitting CG in which the selected recommended product is tried on by the user's alter ego is transmitted to the smartphone 2 of the user 12. Then, according to S291, the transmitted new recommendation history is transmitted to the management server 13 via the store server 6 (S299, S300, S313). The management server 13 additionally registers the received recommendation history (S314).
Next, modified examples of the embodiments described above are listed below.
(1) The object to be tried on is not limited to clothing, but any item that can be worn, such as a ring, a necklace, and a watch.

(2) When performing the fitting simulation, the image of the place desired by the user (customer) 12 may be displayed as a background image. For example, images of cities and tourist spots in Japan and around the world are recorded so that the user (customer) 12 can select and display a desired background.
(3) When updating the virtual division community, the weighting of the user's consumption behavior (trying on or purchasing products) recommended using the action data a updated by machine learning is increased (for example, doubled). It may be controlled so that the virtual division community that reflects the reinforcement learning result is updated. Specifically, referring to FIG. 21a, when both users purchase the product 4 as a result of making recommendations reflecting the reinforcement learning results to the users D and E, the D and E in the user base network K Make the link two lines.
(4) The following invention is disclosed in the above embodiment.

In the case of the fitting simulation method described in Patent Document 1 (Japanese Patent Laid-Open No. 9-106419), the customer only changes the product on the fitting simulation and selects the product, and the use of the fitting simulation is limited.

Therefore, it is conceivable to utilize the fitting simulation for recommending products to customers. For example, the store selects a product that seems to match the customer, tries on the mannequin image of the customer and coordinates it, and sends the fitting image to the customer for recommendation.

However, since the selection of the recommended product, which is the most important part of the recommendation, is made based on the experience and intuition of the clerk, for example, there is a drawback that the scientific selection based on the data is not performed.

The present invention was conceived in view of the actual circumstances, and an object thereof is to enable selection of a product to be scientifically recommended based on data when using a fitting simulation for recommending a product to a customer. Is to do. The present invention employs the following means as means for solving such problems.

It is a fitting simulation system that simulates fitting objects (for example, suits, jackets, shirts, shoes, etc.) by computer graphics for users who wish to try them on.
It is provided with a recommendation means (for example, the flowchart of FIG. 10 or the flowchart of FIGS. 19 and 20 and the agent engine 43 for machine learning) for presenting and recommending a simulation image of trying on a fitting object that seems to match the user. ,
The recommendation means includes a machine learning means (for example, a machine learning agent engine 43) for performing machine learning to acquire knowledge adapted to the user's consumption environment using a fitting simulation.
The machine learning means inputs data that can identify the user's behavior using the fitting simulation as a state (s in FIG. 21B) with respect to the consumption environment (S in FIG. 21B) (for example, s in FIG. 21B). S330 in FIG. 20B), the control that affects the behavior of the user is output as an action on the consumption environment (a in FIG. 21B) (for example, S334 in FIG. 20), and the input and output are output. Reinforcement learning means for performing reinforcement learning to promote the user's consumption behavior by repeating (for example, every time a recommendation request is received in S307 of FIG. 19, a recommendation based on the action a of S308 to S312 is performed, and the result is obtained. It includes updating to action data a that reflects the based state data s (purchase amount of user / number of recommendations) (S334) and performing machine learning by repeating control).
According to the present invention having such a configuration, when the fitting simulation is used for recommending a product to a customer, it is possible to select a product to be scientifically recommended based on the data.

Although the embodiments of the present invention have been described in detail above, the present invention is not limited to these embodiments, and various modifications can be made without departing from the spirit of the present invention.

2 Smartphone 3 Store 5 Data center 6 Store server 8 Tablet terminal 9 Product database 10 Recommendation history database 13 Management server 14 Torso limbs database 16 Member database

Claims (4)

It is a fitting simulation system that simulates the fitting object to be tried on by 3D computer graphics for the user who wants to try it on.
A means of storing multiple types of images of the torso limbs below the head, and a means for storing images of the torso limbs.
An image selection means for selecting an image suitable for the user from the body limb part images stored in the body limb part image storage means, and an image selection means.
A generation means for generating a user's alter ego image by joining the body limb part image selected by the image selection means and a photographed image of the user's head.
An alter ego image storage means for storing the alter ego image generated by the generation means, and
A specific means for identifying the user's alter ego image from the alter ego images stored in the alter ego image storage means, and
A fitting selection receiving means that accepts an operation of selecting a desired fitting object from a plurality of types of fitting object images stored in advance, and a fitting selection receiving means.
Using the image of the object to be tried on selected by the operation received by the fitting selection receiving means, the fitting means on the image for performing a simulation of trying on the alter ego image specified by the specific means on the image, and the fitting means on the image.
Selection control is performed to select a fitting object image that seems to match the alter ego image stored in the alter ego image storage means as a recommendation target, and the recommendation target selected by the selection control is used as the alter ego image. It is provided with a recommendation means for transmitting and recommending a simulation image tried on on the video to a user corresponding to the alter ego image.
The recommendation means is a means for performing the selection control, and stores a plurality of types of fitting object images in association with a priority, and recommends from the plurality of types of fitting object images based on the priority. Including a fitting selection means for selecting a fitting list to be used and having a clerk select a recommendation target from the selected fitting list.
The fitting selection means weights the priority and selects the fitting list as the final priority based on the action data for promoting the consumption behavior of the user updated by the reinforcement learning means.
The reinforcement learning means receives data that can identify the consumption behavior of the user recommended by the recommendation means as state data for the consumption environment, and maximizes the reward in the given consumption environment based on the state data. A fitting simulation system that updates the action data as described above and feeds back the updated action data to the consumption environment. The fitting means on the image performs a simulation in which both the first type fitting object image and the second type fitting object image selected by the operation received by the fitting selection receiving means are tried on the alter ego image. The fitting simulation system according to claim 1, wherein the fitting simulation system can be used. Further equipped with a purchased product storage means for storing information that can identify the purchased product that the user has purchased in the past.
The fitting selection receiving means can accept an operation of selecting from the images of the purchased products of the user specified by the stored information of the purchased product storage means.
In the video fitting means, the purchased product of the user selected by the operation received by the fitting selection receiving means is used as the first type of fitting target image, and the operation received by the fitting selection receiving means. According to claim 2, it is possible to perform a simulation in which both the purchased product and the unpurchased product are tried on the alter ego image by using the unpurchased product of the user selected by the above as the second type of fitting target image. Try-on simulation system. The fitting selection receiving means by the reception is operation is an operation of selecting by the user terminal possessed by the user,
The fitting simulation system according to any one of claims 1 to 3, wherein the fitting means on the image displays the fitting simulation on the display screen of the user terminal.

Images