JP7167738B2 - PRINTING DEVICE, CONTROL METHOD AND COMPUTER PROGRAM - Google Patents

Description

The present invention relates to a technique for printing a design on a sheet in order to attach it to an object on which the design is to be applied.

Conventionally, image forming apparatuses have been designed on the assumption that they will be installed in offices. It is exterior.

Japanese Patent Application Laid-Open No. 2002-132163 JP-A-10-244732 JP-A-9-211923

However, using subdued colors such as white, black, and gray, the exterior image forming apparatus is used in a design such as a color different from that of the image forming apparatus, such as an event venue or a design office. If the image forming apparatus is installed in a room with its own pattern or color, the color of the exterior of the image forming apparatus will not match the pattern or color of the room. Therefore, there is a demand for manufacturing an image forming apparatus having a unique pattern or color on the exterior.

However, in order to meet this demand, when an image forming apparatus is manufactured by incorporating each unique pattern or color into the exterior, an exterior member must be manufactured for each pattern or color, increasing the number of parts. , there is a problem that the manufacturing cost rises.

In order to solve this problem, the present invention provides a printing apparatus, a control method, and a printing apparatus that can make the exterior of an object to be designed incorporate the design drawn on the interior of the area where the object is placed. The purpose is to provide a computer program.

In order to achieve the above object, the present invention provides a printing apparatus for printing a design on a sheet to be pasted on an object to be designed, comprising a flat portion of the exterior surface of the object, or storage means for storing an outer edge shape of a portion that becomes a flat surface by unfolding; design acquisition means for acquiring a design drawn on an interior of an area in which the object is to be placed; generating means for reading a shape, developing the read outer edge shape on a plane, drawing the acquired design in the frame of the developed shape, and generating an exterior image; and printing the exterior image on a sheet. the target object is the printing device; the design acquisition device includes photographing means for generating a photographic image by photography; and when the printing device is installed inside the area, the and extracting means for specifying and extracting one design drawn on the interior of the area in which the printing device, which is the object, is installed from the photographic image generated by the photographing means .

Here, the extracting means extracts one pattern or one color from the photographic image as a design, and the generating means extracts the extracted pattern or one color within a frame of the shape developed on a plane. The color may be painted.

Here, the extracting means may extract the one pattern or the one color occupying the largest area in the photographic image.

Here, the photographic image represents a plurality of patterns or a plurality of colors as a design, and the extraction means occupies a larger area among the plurality of patterns or the plurality of colors represented in the photographic image. A predetermined number of pattern candidates or a predetermined number of color candidates are extracted from the top, and a user designates one pattern or one color from the predetermined number of pattern candidates or the predetermined number of color candidates. An input means for receiving an input may be provided, and the extraction means may extract the one pattern or the one color specified by the input means.

Here, the extraction means extracts one pattern and further comprises an input means for receiving designation of one color by a user's operation, and the generation means receives the color designated by the input means. may be applied within the frame of the shape developed on the plane, and the pattern extracted by the extraction means may be drawn on the applied color.

In this case, further comprising storage means for storing a plurality of color samples respectively representing a plurality of colors, wherein the input means designates one color from among the plurality of color samples by a user's operation. You may accept it.

Here, the extracting means extracts one pattern and further comprises an image reading means for reading a surface to be read, and the generating means generates the above pattern when a color-represented document is placed on the reading surface. The color represented in the read image read by the image reading means is applied in the frame of the shape developed on the plane, and the pattern extracted by the extracting means is drawn on the applied color. may be

Here, the extracting means extracts one color and further comprises an input means for receiving designation of one pattern by user's operation, and the generating means extracts the color extracted by the extracting means. may be applied within the frame of the shape developed on the plane, and the pattern specified by the input means may be drawn on the applied color.

In this case, further comprising storage means for storing a plurality of pattern samples each representing a plurality of patterns, the input means being operated by a user to designate one pattern from among the plurality of pattern samples. You may accept it.

Here, the extracting means extracts one color and further comprises an image reading means for reading a surface to be read, and the generating means develops the color extracted by the extracting means on a plane. When a document with a pattern is applied to the frame of the shape and placed on the reading surface on the application color, the pattern is drawn on the read image read by the image reading means. may be

Here, further comprising input means for receiving designation of the outer edge shape stored in the storage means by user's operation, the generating means may expand the outer edge shape having received the designation on a plane. good.

Here, it is also possible that a portion corresponding to the outer edge shape of the exterior surface of the printing device, which is the object, is provided with a concave portion for pasting the sheet on which the exterior image is printed.

Here, further comprising a first communication means for transmitting the obtained outer edge shape and the design to a spectacles-type display device worn on the user's head for photographing and displaying, the first communication means is a second communication means for receiving the outer edge shape and the design; a photographing means for photographing the surroundings; drawing means for overwriting the received design within a frame of a shape corresponding to the received outer edge shape, and displaying the photographic image overwritten with the design, in the image portion in which the object is represented within. display means; and input means for accepting an input from a user as to whether or not to approve the design represented in the photographic image displayed by the display means, wherein the input means indicates that the design is to be approved. When the input is accepted, the second communication means receives a signal from the spectacles-type display device that transmits an approval signal indicating approval of the design, and when the received signal is the approval signal, the generating means The exterior image may be generated.

Here, when an input to the effect that the design is not approved is received by the input means, the drawing means changes the design to another design, overwrites the changed design on the photographic image, and displays the design. The means displays a photographic image overwritten with the changed design, and the input means further receives from the user whether or not to approve the design represented in the photographic image displayed by the display means. and when the input means receives an input to the effect that the design is approved, the second communication means transmits an approval signal indicating approval of the design and identification information for identifying the changed design. The first communication means receives a signal from the glasses-type display device, and when the received signal includes the approval signal and the identification information, the generation means obtains the design acquired by the design acquisition means. , to the design identified by the received identification information, and drawing the modified design within the frame of the unfolded shape.

Further, the present invention is a control method for controlling a printing device that prints a design on a sheet to be pasted on an object to which the design is to be applied, wherein the printing device controls the exterior surface of the object. Of these, a storage means is provided for storing the outer edge shape of the planar portion or the portion that becomes a planar surface by unfolding, and the control method stores the design drawn on the interior of the area where the object is to be arranged. a step of acquiring a design to be acquired, reading the outer edge shape from the storage means, developing the read outer edge shape on a plane, and drawing the acquired design within the frame of the developed shape to generate an exterior image. and a printing step of printing the exterior image on a sheet , wherein the object is the printing device, and the design acquisition step includes a photographing step of generating a photographic image by photographing; When the printing device is installed inside the area, one design drawn on the interior of the area where the printing device is installed is identified and extracted from the photographic image generated by the photographing step. and an extraction step .

Further, the present invention is a computer program for controlling a printing device for printing a design on a sheet to be pasted on an object to which the design is to be applied, wherein the printing device prints the exterior surface of the object. Among them, a storage means for storing the outer edge shape of the flat portion or the portion that becomes a flat surface by unfolding is provided, and the printing device, which is a computer, is drawn on the interior of the area where the object is to be arranged. a design acquisition step of acquiring the design obtained by reading the outer edge shape from the storage means, developing the read outer edge shape on a plane, drawing the acquired design within the frame of the developed shape, and A generating step of generating an image and a printing step of printing the exterior image on a sheet are executed , the object is the printing device, and the design obtaining step is a photographing step of generating a photographic image by photographing. and identifying, from the photographic image generated by the photographing step, one design drawn on the interior of the area in which the printing device, which is the target object, is installed when the printing device is installed inside the area. and an extraction step of extracting .

According to the above configuration, the exterior of the object to be designed can incorporate the design drawn on the interior of the area where the object is arranged.

1A is a system configuration diagram showing the configuration of an image forming system 1 according to an embodiment; FIG. 2B is a diagram showing a schematic configuration of the image forming apparatus 10; FIG. 4A is a perspective view showing a state in which a sheet 41 is attached to a concave portion 40 formed in the surface of an exterior member 32 of the image forming apparatus 10; FIG. 2(b) is a cross-sectional view taken along the line AA of FIG. 2(a) and showing a cross-section of the exterior member 32 as viewed in the direction of the arrow. 2 is a block diagram showing the configuration of a main control unit 100; FIG. 2 shows the data structure of a menu transition table 211. FIG. (a) shows the data structure of the device list 221; (b) shows an example device image 231; (c) shows another example device image 232; (a) shows the data structure of the exterior member list 241; (b) shows the arrangement of one external component on one device image; 3 shows the data structure of a design list 251. FIG. (a) to (d) show design images 261 to 264 as examples. Exterior scan image 271 is shown as an example. An example design scan image 272 is shown. (a) and (b) show photographic images 295 and 296 as examples. (c) and (d) show identical color blocks 295a and 296a in photographic images 295 and 296; (a) to (c) show exterior images 281 to 283 as examples. An initial menu screen 401, a sticker creation menu screen 411, and a screen 421 representing an apparatus menu displayed on the operation panel 19 are shown. A screen 431 for selecting an exterior member displayed on the operation panel 19, a screen 441 for prompting scanning, and a screen 451 for confirming the scan result are shown. A screen 461 prompting scanning, a screen 471 for confirming the scan result, and a screen 481 representing a design menu displayed on the operation panel 19 are shown. 3 is a block diagram showing the configuration of mobile terminal 300. FIG. A menu screen 341 and a screen 351 for selecting a photo displayed on the mobile terminal 300 are shown. 4 is a flow chart showing an overview of processing from displaying a menu screen to executing printing; 4 is a flow chart showing the operation of exterior selection. 4 is a flow chart showing the operation of exterior scanning. 4 is a flow chart showing the operation of design scanning. 4 is a flow chart showing the operation of design selection; 4 is a flow chart showing an operation of extracting a base color; 4 is a flow chart showing an operation of executing printing; 4 is a flowchart showing an operation of transmitting a photograph from mobile terminal 300 to image forming apparatus 10. FIG. (a) is a system configuration diagram showing the configuration of an image forming system 1a of Modified Example (1). (b) shows an example of an image displayed on the head-mounted display 500; 3 is a block diagram showing the configuration of a head mounted display 500; FIG. 4 is a flowchart showing operations in the image forming system 1a. Continue to FIG. 4 is a flowchart showing operations in the image forming system 1a.

1 Embodiment An image forming system 1 as one embodiment according to the present invention will be described with reference to the drawings.

1.1 Image forming system 1
The image forming system 1, as shown in FIG. 1A, includes an image forming apparatus 10 (printing apparatus) and a portable terminal 300 (photographing apparatus).

The image forming apparatus 10 and the mobile terminal 300 each have a wireless LAN function. Here, the wireless LAN is communication using wireless specified by IEEE802.11a, b, n, ac, etc., for example. The portable terminal 300 captures the interior of the area where the image forming apparatus 10 is installed with the built-in camera by the user's operation, and generates a photographic image. Here, the area is an event venue, a room such as a design office, or the like. Unique designs are drawn on the interior walls, ceilings, floors, surfaces of office furniture, etc. in this area. Here, the design consists of patterns, colors, or a combination of patterns and colors. The mobile terminal 300 transmits the generated photo image to the image forming apparatus 10 via the wireless LAN.

The image forming apparatus 10 receives a photographic image from the mobile terminal 300 via a wireless LAN, and extracts a keynote design (described later) from the received photographic image. The image forming apparatus 10 receives designation of the image forming apparatus 10, another apparatus, or an external member such as an article (target object) from the user. The image forming apparatus 10 acquires the outer edge shape of the designated exterior member, develops the acquired outer edge shape on a plane, draws the extracted basic design in the developed shape, and generates an exterior image. The finished exterior image is printed on the sheet. The user cuts out the exterior image from the sheet (sticker) and pastes the cut exterior image on the image forming apparatus 10, another device, or an exterior member such as an article.

In this way, the image forming apparatus 10 prints the design on the sheet so that it can be attached to the object on which the design is to be applied.

1.2 Image forming apparatus 10
The image forming apparatus 10 is a tandem-type color multifunction peripheral (MFP: MultiFunction Peripheral) having the functions of a scanner, a printer, and a copier.

As shown in FIGS. 1A and 1B, the image forming apparatus 10 is provided with a sheet feeder 13 for storing and feeding sheets in the lower part of the housing. A printer unit 12 (printing means) that forms an image by electrophotography is provided above the paper feed unit 13 . Further above the printer unit 12, an image reader unit 11 (image reading means) that reads the document surface and generates image data, and an operation panel 19 (input means) that displays an operation screen and accepts input operations from the user are provided. It is

As shown in FIG. 1A, the front side of the housing of the image forming apparatus 10 is covered with exterior members 31, 32, 33, 34, 35, . , 34, 35, . . . are each pasted with a sheet having a design. These designed sheets are printed by the image forming apparatus 10, as will be described later.

The image reader section 11 has an automatic document feeder. The automatic document feeder conveys the documents set on the document tray one by one to the document glass plate (reading surface). The image reader unit 11 scans (reads) an image of a document transported to a predetermined position on the document glass plate by the automatic document feeder by moving the scanner, and red (R), green (G), and blue (B) Image data consisting of multilevel digital signals is obtained. The image reader unit 11 writes the obtained image data into the image memory 104 .

The image data of each color component obtained by the image reader unit 11 undergoes various data processing in the main control unit 100, which will be described later. It is converted into image data of each reproduction color.

The printer unit 12 includes an intermediate transfer belt, a drive roller that stretches the intermediate transfer belt, a driven roller, a backup roller, and a plurality of rollers arranged at predetermined intervals along the running direction X of the intermediate transfer belt so as to face the intermediate transfer belt. It consists of an image forming section, a fixing section, a main control section 100, and the like.

Each image forming unit includes a photoreceptor drum as an image carrier, an LED array for exposing and scanning the surface of the photoreceptor drum, a charging charger, a developing device, a cleaner, a primary transfer roller, and the like.

The paper feed unit 13 includes a plurality of paper feed cassettes containing sheets of different sizes, a pickup roller for feeding sheets from each of the paper feed cassettes onto a conveying path, a manual feed tray for placing the sheets, and a manual feed tray. It consists of a pick-up roller for delivering the sheet to the conveying path.

In each imaging station, each photoreceptor drum is uniformly charged by a charging charger and exposed by an LED array to form an electrostatic latent image on the surface of the photoreceptor drum. Each electrostatic latent image is developed by a developing device of each color, and a toner image of Y to K colors is formed on the surface of each photosensitive drum. Due to the electrostatic action of the primary transfer roller, the images are sequentially transferred onto the surface of the intermediate transfer belt.

On the other hand, a sheet is fed from one of the paper feed cassettes or the manual feed tray of the paper feed unit 13 in accordance with the image forming operation of each image forming unit, and the intermediate transfer belt is sandwiched between the secondary transfer roller and the backup roller. is conveyed on the conveying path to the opposing secondary transfer position, and at the secondary transfer position, the toner images of Y to K colors on the intermediate transfer belt are secondarily transferred onto the sheet by the electrostatic action of the secondary transfer roller. be done. The sheet on which the Y to K toner images are secondarily transferred is further conveyed to the fixing section.

The toner image on the surface of the sheet is fused to the surface of the sheet by heat and pressure when passing through the fixing nip formed between the heating roller of the fixing section and the pressure roller in pressure contact therewith. After the sheet passes through the fixing section, the sheet is delivered to a discharge tray.

In this manner, the printer unit 12 prints, for example, an exterior image, which will be described later, on the sheet.

The operation panel 19 is provided with a display screen 20 composed of a liquid crystal display panel or the like, and displays contents set by the user and various messages. The operation panel 19 accepts an instruction to start copying, a setting of the number of copies, a setting of copying conditions, a setting of a data output destination, etc. from the user, and notifies the main control unit 100 of the received contents.

1.3 Exterior Member As shown in FIGS. 2A and 2B, the exterior member 32 that covers the front side of the housing of the image forming apparatus 10 has a box-shaped concave portion 40 on substantially the entire surface of the exterior member 32 . A sheet 41 having the same shape as the bottom surface 40a and having a design is adhered to the rectangular bottom surface 40a of the recess 40 with an adhesive. As shown in FIG. 2B, the depth of the recess 40 is substantially the same as the thickness of the sheet 41. As shown in FIG.

By forming the concave portion 40 in this way, it is possible to minimize the step on the surface of the exterior member 32 when the sheet is attached to the surface of the exterior member 32 .

Other exterior members 31, 33, 34, 35, . The sheet is pasted.

1.4 Main control section 100
As shown in FIG. 3, the main control unit 100 includes a CPU 101, a ROM 102, a RAM 103, an image memory 104, a storage unit 105 (storage means), an image processing unit 106, a network communication unit 107, a printer control unit 108, and a scanner control unit 109. , an input/output unit 110, a display control unit 111, an exterior image processing unit 112, and a wireless communication unit 116 (design acquisition means, reception means, first communication means, second wireless communication means) and the like. The wireless communication section 116 includes a wireless communication control section 113 , a wireless communication circuit 114 and an antenna 115 .

The main control unit 100 controls an image memory 104, a storage unit 105, an image processing unit 106, a network communication unit 107, a printer control unit 108, a scanner control unit 109, an input/output unit 110, according to a scan job, a copy job, a print job, or the like. It controls the display control unit 111, the exterior image processing unit 112, the wireless communication control unit 113, and the like in a unified manner.

(1) CPU 101, ROM 102, RAM 103 and image memory 104
A RAM 103 is composed of a semiconductor memory, temporarily stores various control variables, and provides a work area when the CPU 101 executes a program.

The ROM 102 is composed of a semiconductor memory, and stores in advance control programs and the like for executing various jobs such as scan jobs, copy jobs, and print jobs.

The CPU 101 is composed of a fetch section, a decoding section, an execution section, a register file, an instruction counter, and the like. The fetch unit reads each instruction code included in the control program one by one from the control program stored in the ROM 102 . The decoding unit decodes the read instruction code. The execution part operates according to the decoding result.

The CPU 101 operates according to a control program stored in the ROM 102 to perform image processing unit 106, network communication unit 107, printer control unit 108, scanner control unit 109, input/output unit 110, display control unit 111, external image processing. 112, wireless communication control unit 113, and the like.

For example, when a print job is received by the network communication unit 107 by operating according to a control program, the CPU 101 instructs the printer control unit 108 to execute an image forming operation based on the print job.

The image memory 104 is composed of a semiconductor memory and temporarily stores image data such as a print job.

(2) Image processing unit 106
(color conversion)
For example, the image processing unit 106 processes image data composed of R, G, and B multi-valued digital signals included in a print job received from an external terminal device or generated by scanning by the image reader unit 11. , various data processing is executed, and converted into image data of each reproduction color of Y, M, C, and K. FIG.

(Generation of preview image)
Under the control of the display control unit 111, the image processing unit 106 reduces the image generated by scanning by the image reader unit 11 to generate a low-resolution preview image for preview. Here, the resolution of the preview image is, for example, 50 dpi in the vertical direction and 50 dpi in the horizontal direction. Image processing unit 106 outputs the generated preview image to display control unit 111 . The preview image generated here is used by the user to check the scan result.

(Generation of reduced design image or partial design image)
The image processing unit 106 receives the design ID from the display control unit 111 . Upon receiving the design ID, the image processing unit 106 reads the design image identified by the received design ID from the storage unit 105 . Next, the image processing unit 106 reduces the read design image to generate a reduced design image, or cuts out a portion of the read design image to generate a partial design image. Next, the image processing unit 106 outputs the generated reduced design image or the generated partial design image to the display control unit 111 . The reduced design image or partial design image generated here is used for the user to select a design image.

(Generation of exterior image)
The image processing unit 106 (generating means) receives the shape information (outer edge shape and dimensions) of the exterior member from the later-described generation unit 112b of the exterior image processing unit 112, and acquires the acquired dimensions under the control of the generation unit 112b. The outer edge shape obtained is developed on a plane, and the acquired base color (described later), base pattern (described later), or a combination of the base color and the base pattern is placed in the frame (frame image) of the developed shape. Draw a keynote design and generate an exterior image.

Under the control of the generation unit 112b, the image processing unit 106 extracts the frame (shape of the exterior member) from the exterior scan image (described later) stored in the image memory 104, and the frame is composed of the extracted frame. A frame image is generated on the image memory 104 . Further, the image processing unit 106 draws the acquired basic tone color, basic tone pattern, or basic tone design consisting of a combination of the basic tone color and basic tone pattern in the frame image to generate an exterior image.

The image processing unit 106 outputs the generated exterior image to the printer control unit 108 under the control of the exterior image processing unit 112 . Here, the exterior image is used for printing.

(3) Network communication unit 107, printer control unit 108, and scanner control unit 109
A network communication unit 107 receives a print job from an external terminal device via a network such as a LAN. The network communication unit 107 also outputs messages and the like to external terminal devices as necessary.

The printer control unit 108 uniformly controls the feeding operation from the paper feeding unit 13, the image forming operation of the image forming unit for each reproduction color of the printer unit 12, and the like, and executes the image forming operation.

A scanner control unit 109 controls scanning of the document surface by the image reader unit 11 . For example, the scanner control unit 109 designates the resolution when the image reader unit 11 scans the document surface.

The scanner control unit 109 writes the image data received from the image reader unit 11 into the image memory 104 .

(4) Input/output unit 110
The input/output unit 110 relays data between the display control unit 111 or other control units and the operation panel 19 .

For example, the input/output unit 110 receives menu screens, preview screens, preview images, messages, etc. from the display control unit 111 or other control units, etc., and outputs the received menu screens, preview screens, preview images, messages, etc. to the operation panel. 19.

The input/output unit 110 also receives execution conditions for the image forming apparatus 10 from the operation panel 19, and transmits the received execution conditions to the network communication unit 107, the printer control unit 108, the scanner control unit 109, the display control unit 111, and the like. output.

(5) Wireless communication unit 116
The wireless communication control unit 113 and the wireless communication circuit 114 perform wireless communication based on wireless LAN standards such as IEEE802.11a, b, n, and ac.

An antenna 115 is connected to the wireless communication circuit 114 . Antenna 115 wirelessly transmits and receives radio signals to and from portable terminal 300 . The radio communication circuit 114 performs frequency selection, frequency conversion, and the like of radio signals transmitted and received by the antenna 115 . The wireless communication control unit 113 demodulates the signal received from the wireless communication circuit 114 and outputs it to the exterior image processing unit 112 and the like. Also, the wireless communication control unit 113 modulates a signal received from the exterior image processing unit 112 or the like and outputs the modulated signal to the wireless communication circuit 114 .

Further, the wireless communication unit 116 wirelessly receives a photographic image from the mobile terminal 300, thereby obtaining a design drawn in an area in which an object (for example, the image forming apparatus 10) is to be arranged.

(6) Storage unit 105
The storage unit 105 is composed of a nonvolatile semiconductor memory. Of course, the storage unit 105 may be composed of a hard disk drive.

As shown in FIG. 3, the storage unit 105 stores in advance a menu transition table 211, a first menu screen ID 219, images representing menu screens 401 and 411, images representing screens 441 and 461, screen templates 421a, 431a, 451a, 471a, 481a, device list 221, device images 231, 232, . . . , exterior component list 241, design list 251, design images 261, 262, 263, 264, . A flag 293 and the like are stored. In addition, as shown in FIG. 3, the storage unit 105 includes areas for storing exterior scan images 271, design scan images 272, exterior images 281, 282, 283, . ing.

(Menu transition table 211)
The menu transition table 211 is a data table used to control transition of menu screens displayed on the operation panel 19 .

The menu transition table 211 stores in advance a plurality of menu transition information 212 as shown in FIG. Each menu transition information 212 corresponds to one menu screen.

Each menu transition information 212 includes one menu screen ID 213 and one or more option information 217 .

The menu screen ID 213 is an identifier for uniquely identifying the menu screen.

One option information 217 corresponds to one option displayed on the menu screen.

Therefore, when a plurality of option information 217 is included in one menu transition information 212, the same number of options as the option information 217 are displayed on the menu screen, and the plurality of options in one menu transition information 212 are displayed. Each piece of information 217 corresponds to a plurality of options included in the menu screen.

The option information 217 may include the option ID 214 , the transition destination ID 215 and the function 216 , or may include the option ID 214 and the function 216 without the transition destination ID 215 .

The option ID 214 is an identifier that uniquely identifies the corresponding option.

The transition destination ID 215 is a menu screen ID indicating the menu screen to be displayed next when the option identified by the option ID 214 is selected by the user.

If the transition destination ID 215 does not exist, then there is no menu screen to be displayed next.

Function 216 indicates the function to be performed when the option identified by option ID 214 is selected by the user. Specifically, "transition" indicates transition to the menu screen to be displayed next. "Parameter setting" indicates that parameter settings such as execution conditions are received from the user and stored. "Exterior selection" indicates that selection of an exterior member is accepted from the user. "Design selection" indicates acceptance of design selection from the user. “Exterior scan” indicates that the exterior of an object such as a smartphone is scanned by the image reader unit 11 . “Design scan” indicates that the image reader unit 11 scans the surface of a document on which a design has been applied. “Extraction of base color” indicates extraction of a base color, which will be described later, from the photographic image received from the mobile terminal 300 . “Print execution” indicates that the exterior image with the design is printed on the sheet by the printer unit 12 .

(First menu screen ID 219)
The first menu screen ID 219 is a menu screen ID that identifies the menu screen that should be displayed first on the operation panel 19 immediately after the power of the image forming apparatus 10 is turned on.

(Image representing menu screen 401)
The menu screen 401 is displayed on the display surface 20 of the operation panel 19 immediately after the image forming apparatus 10 is started, and is used by the user to set job execution conditions when executing a copy job.

The image representing the menu screen 401 includes setting areas 402, 403, 404, 405 and a button 406, as shown in FIG.

Setting areas 402, 403, 404, and 405 respectively provide options for setting the density of printing, the paper used for printing, the scaling factor for enlarging or reducing image data when printing, and combining pages when printing. contains. A button 406 is an area that is touched and operated by the user in order to transition to a menu screen 411 (FIG. 13) used for creating a sticker.

(Image representing menu screen 411)
The menu screen 411 is used to select the exterior member of the object, scan the exterior of the object, scan the design, select the design, extract the base color, and print.

The image representing the menu screen 411 includes buttons 412, 413, 414, 415, 416, 417, and 418, as shown in FIG. Buttons 412, 413, 414, 415, 416 are operated by the user to select the object's exterior material, scan the object's exterior, scan the design, select the design, and extract the base color, respectively. This is the area where operations are performed by touching. A button 417 is an area that is touched and operated by the user in order to return to the menu screen 401 . Furthermore, a button 418 indicates an area to be touched and operated by the user in order to execute printing.

(Screen template 421a)
The screen template 421a is a template image based on which the screen 421 shown in FIG. 13 is generated.

The screen template 421a includes, of the screen elements included in the screen 421 shown in FIG.

The title "model name" and the title "model number" are arranged in the device menu area. However, the check box 423, model name 424 as data, and model number 425 as data are not included in the screen template 421a.

A scroll bar 429 is used to scroll through sets of check boxes, model names, and model numbers within the device menu 422 . However, the screen template 421a does not include the knob 426 for scrolling.

A button 427 is operated to cancel the device selection operation on the screen 421 .

A button 428 is operated to confirm the device selection operation on the screen 421 .

(Screen template 431a)
The screen template 431a is a template image based on which the screen 431 shown in FIG. 14 is generated. A screen 431 is used to select an exterior member of the device.

The screen template 431a includes, of the screen elements included in the screen 431 shown in FIG.

No image is placed in the device image area.

A button 434 is operated to cancel the selection operation of the exterior member of the apparatus on the screen 431 .

A button 435 is operated to confirm the selection operation of the exterior member of the apparatus on the screen 431 .

The message 436 includes a description for prompting the user to select an exterior member.

(Image representing screen 441 and screen 461)
A screen 441 shown in FIG. 14 is used when scanning the exterior of the object.

Screen 441 includes message 442 and buttons 443 and 444 .

The message 442 includes a description for prompting the user to execute scanning.

A button 443 is operated to cancel the scan execution operation on the screen 441 .

Button 444 is operated to initiate execution of a scan.

Screen 461 shown in FIG. 15 is similar to screen 441 and includes message 462 and buttons 463 and 464 .

(Screen template 451a)
The screen template 451a is a template image based on which the screen 451 shown in FIG. 14 is generated. A screen 451 is used to display a preview image obtained by scanning the exterior of the object.

The screen template 451a includes, among the screen elements included in the screen 451 shown in FIG. 453 included.

No image is placed in the exterior preview image area.

A button 454 is operated to cancel the scan operation of the exterior of the object on the screen 451 .

A button 455 is operated to confirm the scan operation of the exterior of the object on the screen 451 .

Button 456 is operated to scan the exterior of the object again.

The message 453 includes a description for asking the user to confirm the preview image.

(Screen template 471a)
The screen template 471a is a template image based on which the screen 471 shown in FIG. 15 is generated. Screen 471 is used to display a preview image obtained by scanning the design.

The screen template 471a includes, among the screen elements included in the screen 471 shown in FIG. I'm in.

No image is placed in the design preview area.

A button 474 is operated to cancel the design scanning operation on the screen 471 .

A button 475 is operated to confirm the scanning operation of the design on the screen 471 .

Button 476 is operated to scan the design again.

The message 473 includes a description for asking the user to confirm the preview image.

(Screen template 481a)
The screen template 481a is a template image based on which the screen 481 shown in FIG. 15 is generated.

The screen template 481a includes, of the screen elements included in the screen 481 shown in FIG.

The design menu area does not include a check box 484 and a design 485 as data.

A scroll bar 486 is used to scroll through the set of checkboxes and designs within the design menu 483 . However, the screen template 481a does not include the knob 487 for scrolling.

A button 488 is operated to cancel the design selection operation on the screen 481 .

A button 489 is operated to confirm the design selection operation on the screen 481 .

(Device list 221 and device image)
Device list 221 is a data table used by the user to select a specific device.

The device list 221 includes a plurality of model number information 222 as shown in FIG. 5(a). Each model number information 222 corresponds to one device. Here, each device is an image forming device. However, it is not limited to this. Each device may be a device other than an image forming device, such as a portable communication terminal device, a personal computer, or a home electric appliance.

The model number information 222 includes a model name 223 and a model number 224 .

The model name 223 is a name given to a plurality of devices of the same model.

The model number 224 is an identification number that identifies a plurality of devices that are included in the same model but have different specifications.

A device image 231 shown in FIG. 5B is a reduced image showing the appearance of one device included in the device list 221 . A device image 232 shown in FIG. 5C is also a reduced image showing the appearance of another device, like the device image 231 .

(Exterior material list 241)
The exterior member list 241 is a data table that holds, for all the devices indicated by the device list 221, the outer edge shape, dimensions, and the like of the exterior members of each device.

The exterior member list 241 includes a plurality of pieces of device information 242, as shown in FIG. 6(a). Each device information 242 corresponds to one device.

The device information 242 includes a model number 243 and a plurality of exterior member information 244 . A plurality of pieces of exterior member information 244 respectively correspond to a plurality of exterior members included in one device.

The model number 243 is an identification number that identifies a device of one specification, as described in the device list 221 shown in FIG. 5(a).

The exterior member information 244 is composed of an exterior member number 245 , shape information 246 and arrangement information 247 , and the shape information 246 includes an outer edge shape 248 and dimensions 249 .

The exterior member number 245 is an identification number that identifies the exterior member.

The outer edge shape 248 indicates the shape of a plane portion or a portion that becomes a plane by unfolding on the surface of the exterior member. An example of outer edge shape 248 is a rectangle.

The dimension 249 is, for example, the vertical dimension and the vertical dimension when the outer edge shape 248 is developed on a plane.

The layout information 247 is information indicating the area occupied by the external component on the image representing the external perspective view of the device. The layout information 247 includes the position designated by the user when the user selects one exterior member from one image forming apparatus displayed on the screen 431 of FIG. used to relate

Assume an XY coordinate system in an image representing an external perspective view of the device. For example, an image representing the exterior member 32 is shown in the XY coordinate system shown in FIG. 6B, and the vertices of this image are shown by A11, A12, A13, and A14. At this time, the arrangement information 247 is indicated as XY coordinate values of each of the vertices A11, A12, A13, and A14. Note that in FIG. 6B, description of images of exterior parts other than the image representing the exterior member 32 is omitted.

(design list 251)
The design list 251, as shown in FIG. 7, is a data table holding design IDs 252, which are identification information for identifying all design images stored in the storage unit 105. FIG.

(Design images 261-264)
The design images 261 to 264 are images representing designs, respectively, as shown in FIGS. 8(a) to 8(d).

(exterior scan image 271)
As shown in FIG. 9 , the exterior scan image 271 is, for example, an image generated by reading one side of the smartphone (the side on which the display screen is not arranged) with the image reader unit 11 .

(Design scan image 272)
As shown in FIG. 10, the design scan image 272 is, for example, an image generated by scanning the design from a sheet showing the design with the image reader section 11 .

(Photo images 295, 296)
The photographic images 295 and 296 are photographic images received from the mobile terminal 300, as shown in FIGS. 11(a) and 11(b). The photographic images 295 and 296 are generated by photographing around the position where the image forming apparatus 10 is arranged.

(Exterior images 281-283)
Exterior images 281 and 282 shown in FIGS. 12(a) and 12(b) are, as an example, respectively, the outer edge shapes of the exterior members 32 and 31 of the image forming apparatus 10 shown in FIG. 1(a). 10 is generated by drawing the design scan image 272 shown in FIG. 10 as an example inside the generated frame image.

11C is generated by extracting the frame portion of the smart phone from the scanned exterior image 271 shown in FIG. , is an image generated by drawing the design scan image 272 shown in FIG.

(Exterior flag 291)
The exterior flag 291 is a flag indicating whether the exterior has been selected by the user or whether the exterior has been scanned.

The exterior flag 291 is "0" when the exterior is not selected by the user and the exterior is not scanned.

When the user selects the exterior, the exterior flag 291 is "1".

When the user scans the exterior, the exterior flag 291 is "2".

(Design flag 292)
The design flag 292 is a flag indicating whether the user has selected a design or scanned a design.

Design flag 292 is "0" if no design has been selected by the user and no design has been scanned.

When the user selects a design, the design flag 292 is "1".

If the user scans the design, the design flag 292 is "2".

(Photo flag 293)
Photograph flag 293 is a flag indicating whether or not the base color has been extracted from the photograph image received from portable terminal 300 .

If the base color has not been extracted, the photo flag 293 is "0".

If the base color is extracted, the photo flag 293 is "1".

(7) Display control unit 111
(Screen transition control and screen generation)
The display control unit 111 controls the transition of screens to be displayed on the display surface 20 of the operation panel 19 according to the menu transition table 211 stored in the storage unit 105, as described below.

When image forming apparatus 10 is powered on, display control unit 111 reads first menu screen ID 219 from storage unit 105 . After the menu screen identified by the first menu screen ID 219 is displayed, the display control unit 111 selects from the menu transition table 211 a transition destination ID (menu screen ID) that identifies the menu screen to be displayed next. read out.

The display control unit 111 reads the menu screen identified by the read menu screen ID from the storage unit 105, outputs the read menu screen to the operation panel 19 via the input/output unit 110, and reads it. A menu screen is displayed on the operation panel 19 .

The display control unit 111 reads menu transition information including the read menu screen ID from the menu transition table 211 .

When the user operates the menu screen being displayed, the display control unit 111 determines the content of the function in the option information included in the read menu transition information. Specifically, the display control unit 111 determines that the contents of the functions are "parameter setting", "transition", "exterior selection", "exterior scanning", "design scanning", "design selection", and "extraction of base color". ', 'run print' and other. Next, the display control unit 111 operates according to the functions in the option information.

If the function in the option information is "transition", the display control unit 111 extracts the corresponding transition destination ID (menu screen ID) from the option information, and includes the menu screen ID corresponding to the extracted transition destination ID. Menu transition information is read from the menu transition table 211 . Further, the display control unit 111 reads the menu screen corresponding to the transition destination ID from the storage unit 105, outputs the read menu screen to the operation panel 19 via the input/output unit 110, and displays the read menu screen. A screen is displayed on the operation panel 19 .

For example, the display control unit 111 displays menu screens 401 and 411 shown in FIG. 13 on the display surface 20 of the operation panel 19 .

Further, after displaying the menu screen 411, the display control unit 111 executes exterior selection, exterior scanning, design scanning, design selection, or base color extraction processing. A menu screen 411 in which the buttons 412, 413, 414, 415, and 416 corresponding to the executed processing are reversed is generated and generated so that the user can clearly know that the processing has been executed. Menu screen 411 may be output to operation panel 19 via input/output unit 110 and menu screen 411 may be displayed on operation panel 19 .

(Generation and display control of screen 421)
If the function in the option information is “exterior selection”, display control unit 111 reads screen template 421 a from storage unit 105 . Next, the display control unit 111 reads all the model number information from the device list 221 stored in the storage unit 105, and for all the read model number information, displays the image of the check box, the model name and the model name included in the model number information. A set of model numbers is generated, and a device menu incorporating the generated sets is generated. Note that when the number of sets included in the device menu exceeds a predetermined number, the display control unit 111 incorporates only a predetermined number of sets out of the plurality of generated sets into the device menu. Next, the display control unit 111 arranges the device menu in the device menu area within the screen template 421a. Also, the display control unit 111 arranges the knob 426 on the scroll bar 429 .

In this manner, the display control unit 111 generates a screen 421 (FIG. 13) for selecting a device, and outputs the generated screen 421 to the operation panel 19 via the input/output unit 110. to display the generated screen 421 on the display surface 20 of the operation panel 19 .

Note that when the screen 421 is displayed, the display control unit 111 replaces the group to be incorporated into the device menu by the user's operation of the knob 426 .

When the user selects a set of model name and model number from the device menu 422 and confirms the selection by operating the confirmation button 428, the display control unit 111 confirms the selection by the user. A set of model name and model number is written in the storage unit 105 .

(Display control of screen 441)
If the function in the option information is “external scan”, the display control unit 111 reads the screen 441 (FIG. 14) for prompting scanning by the image reader unit 11 from the storage unit 105, and inputs the read screen 441. The generated screen 441 is output to the operation panel 19 via the output unit 110 and displayed on the display surface 20 of the operation panel 19 .

(Display control of screen 461)
If the function in the option information is "design scan", the display control unit 111 reads the screen 461 (FIG. 15) for prompting scanning by the image reader unit 11 from the storage unit 105, and inputs the read screen 461. The generated screen 461 is output to the operation panel 19 via the output unit 110 and displayed on the display surface 20 of the operation panel 19 .

(Generation and display control of screen 481)
If the function in the option information is “design selection”, display control unit 111 reads screen template 481 a from storage unit 105 . Next, display control section 111 reads out all design IDs from design list 251 stored in storage section 105 . Next, the display control unit 111 outputs all the read design IDs to the image processing unit 106, and the image processing unit 106 processes a reduced design image obtained by reducing the design image identified by each design ID or each design. The image processing unit 106 is controlled so as to generate a partial design image obtained by cutting out a portion of the design image identified by the ID. Next, the display control unit 111 receives reduced design images or partial design images from the image processing unit 106 for all read design IDs.

Next, the display control unit 111 generates a set of check box images and reduced design images or partial design images, and generates a design menu incorporating the generated sets. Note that when the number of sets included in the design menu exceeds a predetermined number, the display control unit 111 incorporates only a predetermined number of sets out of the generated sets into the design menu. Next, the display control unit 111 arranges the design menu in the design menu area within the screen template 481a. Also, the display control unit 111 arranges the knob 487 on the scroll bar 486 .

In this manner, the display control unit 111 generates a screen 481 (FIG. 15) for design selection, and outputs the generated screen 481 to the operation panel 19 via the input/output unit 110. to display the generated screen 481 on the display surface 20 of the operation panel 19 .

Note that when the screen 481 is displayed, the display control unit 111 replaces the set to be incorporated into the design menu by the user's operation of the knob 487 .

When a design is selected by the user and confirmed by operating the confirmation button 489, the display control unit 111 writes a design ID for identifying the design selected by the user to the storage unit 105, and the design is confirmed. Flag 292 is set to "1".

(Generation and display control of screen 431)
When the confirmation button 428 is operated while the screen 421 (FIG. 13) for selecting the device is displayed, the display control unit 111 selects the exterior member as follows. screen 431 (FIG. 14).

The display control unit 111 reads the screen template 431 a from the storage unit 105 . Next, the display control unit 111 reads the model number of the device selected by the user on the screen 421 from the storage unit 105, and reads the device image identified by the read model number from the storage unit 105. FIG. Next, the display control unit 111 arranges the read device image in the device image area of the screen template 431a. Thus, the display control unit 111 generates the screen 431. FIG. Next, the display control unit 111 outputs the generated screen 431 to the operation panel 19 via the input/output unit 110 to display the generated screen 431 on the display surface 20 of the operation panel 19 .

The display control unit 111 receives, from the operation panel 19, the display positions of one or more exterior members selected by the user from among the plurality of exterior members displayed in the device image displayed on the screen 431. . Based on the received display position of the exterior member and the arrangement information (FIG. 6(a)) in the exterior member information, the exterior member selected by the user is specified, and the identification member number for identifying the exterior member is Extract from the exterior member information.

When the user selects an exterior member of the apparatus and the selection is confirmed by operating the confirmation button 435, the display control unit 111 identifies the exterior member selected by the user. The number is written in the storage unit 105, and the exterior flag 291 is set to "1".

(Generation and display control of screen 451)
When the scan execution button 444 is operated while the screen 441 ( FIG. 14 ) for performing scanning is displayed, the display control unit 111 sends the image data generated by scanning to the image reader unit 11 ( exterior scan image) is written in the image memory 104 . Further, the image processing unit 106 is controlled to reduce an image generated by scanning by the image reader unit 11 and generate a preview image. Next, the display control section 111 receives the preview image from the image processing section 106 . Also, the display control unit 111 reads out the screen template 451 a from the storage unit 105 . Next, the display control unit 111 arranges the received preview image in the exterior preview image area of the screen template 451a to generate the screen 451 (FIG. 14). Next, the display control unit 111 outputs the generated screen 451 to the operation panel 19 via the input/output unit 110 to display the generated screen 451 on the display surface 20 of the operation panel 19 .

When the user scans the exterior of the object and operates the confirmation button 455 to confirm, the exterior flag 291 is set to "2".

(Generation and display control of screen 471)
When the scan execution button 464 is operated while the screen 461 (FIG. 15) for prompting the scan is displayed, the display control unit 111 sends the image data generated by the scan to the image reader unit 11 ( The design scan image) is controlled to be written in the image memory 104 . The display control unit 111 also controls the image processing unit 106 to reduce an image generated by scanning by the image reader unit 11 and generate a preview image. Next, the display control section 111 receives the preview image from the image processing section 106 . Also, the display control unit 111 reads out the screen template 471 a from the storage unit 105 . Next, the display control unit 111 arranges the received preview image in the design preview image area of the screen template 471a to generate the screen 471 (FIG. 15). Next, the display control unit 111 outputs the generated screen 471 to the operation panel 19 via the input/output unit 110 to display the generated screen 471 on the display surface 20 of the operation panel 19 .

When the user scans the design and operates the confirmation button 475 to confirm the design, the display control unit 111 sets the design flag 292 to "2".

(Acceptance of execution conditions)
If the function in the option information is “parameter setting”, display control unit 111 accepts execution conditions set by the user from operation panel 19 and writes the received execution conditions into storage unit 105 .

For example, as shown in the menu screen 401 of FIG. 13, when one of options indicating paper selection is selected, the display control unit 111 sets the paper size corresponding to the selected option (or Identifier for identifying the size) is written in the storage unit 105 as the paper size as one of the execution conditions.

At this time, in order for the user to clearly know that the option has been selected, a screen in which the received setting item is highlighted is generated as necessary, and the generated screen is sent to the input/output unit 110. may be output to the operation panel 19 via , and the screen may be displayed on the operation panel 19 .

(8) Exterior image processing unit 112
The exterior image processing unit 112 includes, as shown in FIG. 3, an extraction unit 112a (extraction means) and a generation unit 112b (generation means).

(Extraction processing of base color from photographic image)
When the button 416 for extracting the base color is operated while the menu screen 411 (FIG. 13) is displayed, the extraction unit 112a executes the processing for extracting the base color from the photographic image as follows. do.

The extraction unit 112a reads out all photographic images from the storage unit 105 . Next, the extraction unit 112a divides the inside of each photographic image into blocks for each color. Here, for example, as shown in FIG. 11C, one block 295a is divided and cut out from the photograph image 295, and one block 295a is cut out from the photograph image 296 as shown in FIG. Block 296a is divided and cut out. Blocks 295a and 295b are wall surfaces in photographic images 295 and 296, and are expressed in the same color.

Next, the extraction unit 112a calculates the area of each block and stores the calculated area. Here, for example, the areas of blocks 295a and 296a shown in FIGS. 11(c) and 11(d) are calculated.

Next, the extracting unit 112a sums up the areas of the blocks for each color with respect to all the photographic images of the exterior. Here, for example, the areas of blocks 295a and 296a shown in FIGS. 11(c) and 11(d) are totaled.

Next, the extraction unit 112a calculates the maximum area among the areas of each color, selects the color corresponding to the maximum area as one base color, and identifies the selected base color. Store information.

Alternatively, the extraction unit 112a may extract one basic pattern as follows.

The extraction unit 112a identifies one pattern in the photographic image, and identifies an area in which the same pattern as the identified pattern is drawn. Identification of an area in which the same pattern as the identified pattern is drawn is repeated for all areas in the photographic image. Next, the extracting unit 112a calculates the sum of the areas of all regions where the same pattern as the specified pattern is drawn. The exterior image processing unit 112 calculates the total area of all areas in which the same pattern is drawn, even for other patterns, in the photographic image. Next, the extracting unit 112a calculates the maximum area among the areas of the patterns, selects the pattern corresponding to the maximum area as the base pattern, and extracts identification information indicating the selected base pattern. Remember.

In addition, when the basic tone pattern is drawn on the basic tone color, the extraction unit 112a may extract the basic tone design composed of the basic tone color and the basic tone pattern.

Here, the design included in the photographic image includes at least one pattern or at least one color. Also, the design included in the photographic image may include multiple patterns or multiple colors.

As described above, the extraction unit 112a extracts from the photographic image a basic tone design composed of a basic tone color, a basic tone pattern, or a combination of a basic tone color and a basic tone pattern.

(Generation of exterior image)
When the print execution button 418 is operated while the menu screen 411 (FIG. 13) is displayed, the generating unit 112b generates an exterior image as described below.

The generation unit 112b determines whether the exterior flag 291 is "0", "1", or "2", and determines whether the design flag 292 is "0", "1", or "2". It determines whether the photo flag 293 is "0" or "1".

If the exterior flag 291 is "0" or the design flag 292 is "0", the exterior or design of the object is not specified, respectively. , the exterior image may not be generated.

When the exterior flag 291 is "1", the generation unit 112b reads from the storage unit 105 the model number of the device selected by the user and the exterior member number of the exterior member selected by the user. Next, the generation unit 112b reads shape information (outer edge shape and dimensions) corresponding to the read model number and the exterior member from the exterior member list 241 . Next, the generation unit 112b outputs the read shape information to the image processing unit 106, controls the image processing unit 106, and generates a frame image in which the outer edge shape is developed on a plane on the image memory 104. Let

In this way, the generation unit 112b reads the outer edge shape from the storage unit 105, develops the read outer edge shape on a plane, and stores the acquired design (color, pattern, or color and pattern combination) in the developed shape. ) to generate an exterior image.

When the exterior flag 291 is "2", the generation unit 112b causes the image processing unit 106 to extract the frame portion from the exterior scan image stored in the image memory 104, and draws the extracted frame portion. A frame image is generated on the image memory 104 .

When the design flag 292 is "1", the generation unit 112b reads from the storage unit 105 a design ID that identifies the design image selected by the user. Next, the generation unit 112b causes the image processing unit 106 to read the design image identified by the read design ID from the storage unit 105 onto the image memory 104, and creates a pattern from the read design image. Let it be extracted. Next, when the photo flag is “1”, the generation unit 112b controls the image processing unit 106 to apply the base color within the frame image generated on the image memory 104 . Next, the generation unit 112b controls the image processing unit 106 to draw the extracted pattern on the frame image or on the frame image applied with the base color to generate the exterior image.

When the design flag 292 is "2", the generation unit 112b controls the image processing unit 106 to read out the design scan image from the storage unit 105 onto the image memory 104, and , to extract the pattern. Next, when the photo flag is "1", the generation unit 112b controls the image processing unit 106 to apply the base color within the frame image. Next, the generation unit 112b controls the image processing unit 106 to draw the extracted pattern on the frame image or on the frame image applied with the base color to generate the exterior image.

After finishing the generation of the exterior image, the generation unit 112b controls the printer control unit 108 to print the exterior image.

1.5 Transition of Screens Transitions of screens displayed on the display surface 20 of the operation panel 19 will be described below.

A menu screen 401 shown in FIG. 13 is displayed on the display surface 20 of the operation panel 19 immediately after the image forming apparatus 10 is started.

When the user operates a button 406 indicating creation of a sticker on the menu screen 401, a menu screen 411 is displayed next.

Next, when the button 412 indicating exterior selection is operated on the menu screen 411, a screen 421 is displayed. When the user selects a set of model name and model number on the screen 421 and operates a button 428 indicating confirmation, then a screen 431 shown in FIG. 14 is displayed. When the user selects an exterior member on the screen 431 and operates the button 435 indicating confirmation, the menu screen 411 is displayed next.

Further, when the button 413 indicating exterior scanning is operated on the menu screen 411, a screen 441 shown in FIG. 14 is displayed next. For example, when the user places the smartphone face down on the document glass plate of the image reader unit 11 and operates the button 444 indicating execution of scanning, a screen 451 is displayed next. When the user operates a button 455 indicating confirmation on the screen 451, the menu screen 411 is displayed next.

Further, when the button 414 indicating design scan is operated on the menu screen 411, a screen 461 shown in FIG. 15 is displayed next. When the user places, for example, a sheet showing a design on the document glass plate of the image reader unit 11 and operates the button 464 indicating execution of scanning, a screen 471 is displayed next. When the user operates a button 475 indicating confirmation on the screen 471, the menu screen 411 is displayed next.

Further, when the button 415 indicating design selection is operated on the menu screen 411, a screen 481 shown in FIG. 15 is displayed next. When the user selects a design and operates the button 489 indicating confirmation, the menu screen 411 is displayed next.

Next, when a button 418 indicating print execution is operated on the menu screen 411, the exterior image is printed on the sheet according to the set outer edge shape and design.

1.6 Mobile terminal 300
As shown in FIG. 16, the mobile terminal 300 (photographing device) includes, for example, a main control unit 301, a storage unit 302, an audio processing unit 306, a speaker 307, a microphone 308, an input/output control unit 309, a touch panel 310, an input unit 311, a second wireless communication unit 315 (output means, first wireless communication means), a first wireless communication unit 316, a camera input unit 317 and a camera 318 (photographing means). The second wireless communication section 315 has an antenna 312 , a second communication circuit 313 and a second communication control section 314 . The first wireless communication section 316 has an antenna 304 , a first communication circuit 303 and a first communication control section 305 .

The mobile terminal 300 is specifically a computer system including a microprocessor, a signal processor, a ROM, a RAM, and the like. A computer program is stored in the RAM. The microprocessor and signal processor operate according to a computer program. Thereby, the main control unit 301, the audio processing unit 306, the input/output control unit 309, the second communication control unit 314, the first communication control unit 305, and the camera input unit 317 achieve their functions.

Portable terminal 300 is, for example, a smartphone. Note that the mobile terminal 300 may be a touch pad, a mobile phone, a mobile personal computer, or the like.

(1) Storage unit 302
The storage unit 302 is configured by, for example, a nonvolatile semiconductor memory.

The storage unit 302 has areas for storing photographic images 331, 332, . . .

(2) Antenna 304, first communication circuit 303 and first communication control section 305
The antenna 304, the first communication circuit 303, and the first communication control unit 305 realize mobile communication based on standards such as CDMA (Code Division Multiple Access) 2000 and LTE (Long Term Evolution).

Antenna 304 transmits and receives radio signals to and from a radio base station via a radio channel. The first communication circuit 303 performs frequency selection, frequency conversion, etc. of radio signals transmitted and received by the antenna 304 . The first communication control section 305 demodulates the signal received from the first communication circuit 303 and outputs it to the main control section 301 or the audio processing section 306 . The first communication control unit 305 also modulates a signal received from the main control unit 301 or the audio processing unit 306 and outputs the modulated signal to the first communication circuit 303 .

(3) Antenna 312, Second Communication Circuit 313, and Second Communication Control Unit 314
The antenna 312, the second communication circuit 313, and the second communication control unit 314 perform wireless communication based on wireless LAN standards such as IEEE802.11a, b, n, and ac.

The antenna 312 wirelessly transmits and receives wireless signals to and from the image forming apparatus 10 . The second communication circuit 313 performs frequency selection, frequency conversion, etc. of radio signals transmitted and received by the antenna 312 . The second communication control section 314 demodulates the signal received from the second communication circuit 313 and outputs it to the main control section 301 . The second communication control unit 314 also modulates the signal received from the main control unit 301 and outputs the modulated signal to the second communication circuit 313 .

The second communication control unit 314 receives an instruction from the main control unit 301 to transmit the photographic image stored in the storage unit 302 to the image forming apparatus 10 . Upon receiving the transmission instruction, the second communication control unit 314 further receives the photographic image from the main control unit 301 and transmits the received photographic image to the image forming apparatus 10 via the second communication circuit 313 and the antenna 312. Send to

(4) Audio processor 306, microphone 308 and speaker 307
The microphone 308 generates an audio signal as an analog electrical signal by vibrating a coil in a magnetic field with a diaphragm that vibrates upon receiving sound waves. The sound wave received by the microphone 308 is, for example, the user's voice. Next, the generated audio signal is output to the audio processing unit 306 .

The speaker 307 receives an audio signal as an analog electrical signal from the audio processing unit 306, vibrates the diaphragm according to the received audio signal, and outputs sound waves.

The audio processing unit 306 decodes encoded audio information obtained by encoding audio information, further converts the decoded audio information into an analog electrical signal, and generates an audio signal. Next, audio processor 306 outputs the generated audio signal to speaker 307 . Also, the audio processing unit 306 converts the audio signal received from the microphone 308 into a digital electrical signal and encodes it to generate encoded audio information. Next, the audio processing section 306 outputs the generated encoded audio information to the first communication control section 305 .

(5) Camera input unit 317 and camera 318
The camera 318 is installed on the back surface of the operation surface of the mobile terminal 300 so that its optical axis faces forward in a direction perpendicular to the back surface of the operation surface of the mobile terminal 300, and the shooting direction is fixed.

The camera 318 includes a lens that collects light from a subject and forms an image on an imaging element, an imaging element such as a CCD (Charge Coupled Device) that converts the irradiated light into an electrical signal through the lens, and an electrical signal that is generated. and an image processing engine that converts the image into digital image data. The camera 318 outputs a photographic image (image data) generated by the image processing engine to the camera input unit 317 .

The horizontal angle of view of the camera 318 is, for example, approximately 90 degrees, and the vertical angle of view is, for example, approximately 45 degrees.

The camera input unit 317 receives photographic images from the camera 318 and writes the received photographic images into the storage unit 302 as photographic images 331, 332, and the like.

(6) Touch panel 310, input unit 311 and input/output control unit 309
The touch panel 310 includes a display panel section having a rectangular display surface and a touch pad section attached to the display surface. The display panel section is, for example, a liquid crystal display. The touch pad unit detects contact of an operation object such as a user's finger on the operation surface of the touch panel 310 .

As shown in FIG. 17, the touch panel 310 displays, as an example, a menu screen 341 and a screen 351 for selecting a photo.

A plurality of icons for operation are displayed on the menu screen 341 . The icons include an icon 342 for selecting and sending a photo. When icon 342 is operated by the user, touch panel 310 displays screen 351 .

The screen 351 includes a selection list 352 for selecting a photo image and a button 353 for instructing transmission of the photo. When the user selects one or a plurality of photographic images from the selection list 352 and operates the button 353, the selected photographic images are wirelessly transmitted from the portable terminal 300 to the image forming apparatus 10. sent.

The input unit 311 has a plurality of buttons. Each button is operated by being pressed by the user.

The input/output control unit 309 receives an operation signal from the input unit 311 as an input signal. One of the operation signals indicates an operation for camera 318 . Also, position information is received from the touch panel 310 as an input signal. The input/output control unit 309 outputs the received input signal to the main control unit 301 . The input/output control unit 309 also receives characters and images from the main control unit 301 and controls the touch panel 310 to output and display the received characters and images.

(7) Main control section 301
The main control unit 301 controls the storage unit 302 , first communication control unit 305 , audio processing unit 306 , input/output control unit 309 , second communication control unit 314 and camera input unit 317 .

The main control unit 301 receives from the input/output control unit 309 an input signal indicating a photographing operation by the camera 318 and an input signal indicating an instruction to transmit a photographic image to the image forming apparatus 10 .

Upon receiving an input signal indicating a shooting operation from the input/output control unit 309 , the main control unit 301 controls the camera input unit 317 so that the camera 318 shoots. Main control unit 301 also receives a photographic image from camera input unit 317 . When the photographic image is received, the received photographic image is written in the storage unit 302. - 特許庁

Further, when an input signal indicating an instruction to transmit a photographic image to the image forming apparatus 10 is received from the input/output control unit 309 , the main control unit 301 instructs the second wireless communication unit 315 to store the image in the storage unit 302 . instructing the image forming apparatus 10 to transmit the stored photographic image, reading the photographic image to be transmitted from the storage unit 302, and transmitting the read photographic image to the second wireless communication unit 315; Output.

1.6 Operation of Image Forming System 1 Operation of the image forming system 1 will be described.

(1) General Operations of Image Forming Apparatus 10 General operations of the image forming apparatus 10 from displaying the menu screen 401 to executing printing will be described with reference to the flowchart shown in FIG.

The display control unit 111 generates an initial menu screen (step S101), and the operation panel 19 displays the menu screen (step S102). The operation panel 19 accepts the selection of options by the user's operation (step S103).

The display control unit 111 determines the content of the received option (step S104). If the content of the option is to display the next menu screen, or if it is "return" ("display next menu screen or return" in step S104), the display control unit 111 generates the next menu screen. (step S105). Next, the display control unit 111 shifts the control to step S102.

If the content of the option is exterior selection ("exterior selection" in step S104), exterior selection processing is performed (step S106), and then display control unit 111 shifts the control to step S105.

If the content of the option is external scan ("external scan" in step S104), external scan is performed (step S107), and then display control unit 111 shifts the control to step S105.

If the content of the option is design scan ("design scan" in step S104), design scan is performed (step S108), and then display control unit 111 shifts the control to step S105.

If the content of the option is design selection ("design selection" in step S104), design selection is performed (step S109), and then display control unit 111 shifts the control to step S105.

If the content of the option is to extract the base color (“Extract base color” in step S104), the base color is extracted from the photograph (step S110). to

If the content of the option is print execution ("print execution" in step S104), printing is executed (step S111), and then the display control unit 111 shifts the control to step S101.

If the content of the option is other ("other" in step S104), other processing is executed (step S112), and then the display control unit 111 shifts the control to step S101.

This concludes the description of the general operation of the image forming apparatus 10 .

(2) Exterior Selection Operation The exterior selection operation of the image forming apparatus 10 will be described with reference to the flowchart shown in FIG. Note that the operation described here is the details of step S106 in FIG.

The display control unit 111 reads all model number information from the device list 221 stored in the storage unit 105 (step S201).

Next, the display control unit 111 generates a set of check box images, a model name and a model number included in the model number information, and generates an apparatus menu consisting of the plurality of generated sets. (Step S202). Next, the display control unit 111 arranges the generated device menu in the device menu area in the screen template 421a to generate a screen 421 for selecting a device, and the operation panel 19 displays the screen 421 ( step S203).

The operation panel 19 accepts selection of a set of model name and model number from the device menu 422 on the screen 421 by the user (step S204). The display control unit 111 determines the user's operation (step S205), and if the user's operation is "cancel" ("cancel" in step S205), the display control unit 111 ends the operation of selecting the exterior. do.

If the user's operation is "Confirm" ("Confirm" in step S205), the display control unit 111 reads the device image identified by the model number of the device selected by the user from the storage unit 105 (step S206). ). Next, the display control unit 111 generates a screen 431 including the read device image (step S207). Next, the operation panel 19 displays the screen 431 (step S208).

The operation panel 19 accepts the selection of the exterior member of the apparatus from the user (step S209). Here, the operation panel 19 may accept selection of a plurality of exterior members from the user. The display control unit 111 determines the user's operation (step S210), and if the user's operation is "cancel" ("cancel" in step S210), the display control unit 111 ends the exterior selection operation. do.

If the user's operation is "Confirm" ("Confirm" in step S210), the display control unit 111 writes the exterior member number identifying the exterior member selected by the user into the storage unit 105 (step S211). . Next, the display control unit 111 reverses the exterior display options (step S212). Next, the display control unit 111 sets the exterior flag 291 to "1" (step S213).

Thus, the exterior selection operation by the image forming apparatus 10 is completed.

(3) Exterior scan operation Exterior scan operation by the image forming apparatus 10 will be described with reference to the flowchart shown in FIG. 20 . Note that the operation described here is the details of step S107 in FIG.

The display control unit 111 reads the screen 441 for prompting scanning by the image reader unit 11 from the storage unit 105 (step S231). Next, the operation panel 19 displays the screen 441 (step S232). Here, the user places the smartphone on the document glass plate of the image reader unit 11 with one surface facing downward.

With the screen 441 displayed, the display control unit 111 determines the user's operation (step S233). When the cancel button 443 is operated (“Cancel” in step S233), the display control unit 111 terminates the exterior scan operation.

When the scan execution button 444 is operated ("scan execution" in step S233), the image reader unit 11 executes scanning and writes the exterior scanned image generated by the scanning to the storage unit 105 (step S234). The image processing unit 106 reduces the image generated by scanning by the image reader unit 11 to generate a preview image (step S235). Next, the display control unit 111 arranges the preview image in the exterior preview image area of the screen template 451a to generate the screen 451 (step S236). Next, the operation panel 19 displays the screen 451 (step S237).

With the screen 451 displayed, the display control unit 111 determines the user's operation (step S238). When the cancel button 454 is operated (“Cancel” in step S238), the display control unit 111 terminates the exterior scanning operation.

When the confirm button 455 is operated (“confirm” in step S238), the display control unit 111 reverses the exterior scan options (step S239). Next, the display control unit 111 sets the exterior flag 291 to "2" (step S240).

As described above, the exterior scanning operation by the image forming apparatus 10 is completed.

(4) Operation of Design Scan The operation of design scan by the image forming apparatus 10 will be described with reference to the flowchart shown in FIG. Note that the operation described here is the details of step S108 in FIG.

The display control unit 111 reads the screen 461 for prompting scanning by the image reader unit 11 from the storage unit 105 (step S251). Next, the operation panel 19 displays the screen 461 (step S252).

With the screen 461 displayed, the display control unit 111 determines the user's operation (step S253). When the cancel button 463 is operated (“cancel” in step S253), the display control unit 111 terminates the design scan operation.

When the scan execution button 464 is operated (“Execute scan” in step S253), the image reader unit 11 executes scanning, and writes the design scan image generated by scanning into the storage unit 105 (step S254). The image processing unit 106 reduces the image generated by scanning by the image reader unit 11 to generate a preview image (step S255). Next, the display control unit 111 arranges the preview image in the design preview image area of the screen template 471a to generate the screen 471 (step S256). Next, the operation panel 19 displays the screen 471 (step S257).

With the screen 471 displayed, the display control unit 111 determines the user's operation (step S258). When the cancel button 474 is operated (“Cancel” in step S258), the display control unit 111 terminates the design scan operation.

When the confirm button 475 is operated (“Confirm” in step S258), the display control unit 111 reverses the design scan option (step S259). Next, the display control unit 111 sets the design flag 292 to "2" (step S260).

Thus, the operation of design scanning by the image forming apparatus 10 is completed.

(5) Design Selection Operation The design selection operation by the image forming apparatus 10 will be described with reference to the flowchart shown in FIG. Note that the operation described here is the details of step S109 in FIG.

The display control unit 111 reads all design IDs from the design list 251 stored in the storage unit 105 (step S271). Next, the image processing unit 106 reads design images identified by all read design IDs (step S272). Next, the image processing unit 106 generates a reduced design image obtained by reducing the read design image or a partial design image obtained by cutting out a part of the read design image (step S273). Next, the display control unit 111 generates the screen 481 by generating a design menu including the check box image and the reduced design image or the partial design image and arranging the design menu in the design menu area within the screen template 481a. (step S274). The operation panel 19 displays the screen 481 (step S275).

The operation panel 19 accepts selection of a partial design image or reduced design image from the user (step S276).

With the screen 481 displayed, the display control unit 111 determines the user's operation (step S277). When the cancel button 488 is operated (“Cancel” in step S277), the display control unit 111 terminates the design selection operation.

When the confirm button 489 is operated (“Confirm” in step S277), the display control unit 111 writes the design ID corresponding to the selected partial design image or reduced design image into the storage unit 105 (step S278). Next, the display control unit 111 reverses the options for design selection (step S279). Next, the display control unit 111 sets the design flag 292 to "1" (step S280).

Thus, the design selection operation by the image forming apparatus 10 is completed.

(6) Operation of Extracting Base Color from Photo Image The operation of extracting the base color from the photo image by the image forming apparatus 10 will be described with reference to the flowchart shown in FIG. Note that the operation described here is the details of step S110 in FIG.

The exterior image processing unit 112 reads one photographic image from the storage unit 105 (step S291). Next, exterior image processing unit 112 divides the read photographic image into blocks for each color (step S292).

Next, the exterior image processing unit 112 calculates the area of each block (step S293), and stores the calculated area of each block (step S294).

If reading of all photographic images stored in storage unit 105 has not been completed ("NO" in step S295), exterior image processing unit 112 repeats steps S291 to S294.

When all the photographic images stored in the storage unit 105 have been read out (“YES” in step S295), the exterior image processing unit 112 sums up the block area for each color of all the photographic images. (step S296).

Next, the exterior image processing unit 112 calculates the maximum area among the areas of each color, selects the color corresponding to the maximum area as the base color, and identifies the selected base color. The information is stored (step S297).

Next, the display control unit 111 reverses the options for extracting the base color (step S298). Next, the display control unit 111 sets the photo flag 293 to "1" (step S299).

Thus, the operation of extracting the base color from the photographic image by the image forming apparatus 10 is completed.

(7) Print execution operation The print execution operation will be described with reference to the flowchart shown in FIG. Note that the operation described here is the details of step S111 in FIG.

If the exterior flag 291 is "0" or the design flag 292 is "0" ("YES" in step S300), the exterior member of the object is not specified, or the design is specified. Therefore, the exterior image processing unit 112 terminates without executing printing.

When exterior flag 291 is not "0" and design flag 292 is not "0" ("NO" in step S300), exterior image processing unit 112 determines when exterior flag 291 is "1" (step S301). "1" in step S302), the exterior image processing unit 112 reads the model number of the apparatus selected by the user and the exterior member number of the exterior member selected by the user from the storage unit 105 (step S302). Next, the exterior image processing unit 112 reads shape information corresponding to the read model number and exterior member from the exterior member list 241 (step S303). Next, the image processing unit 106 generates a frame image in which the outer edge shape of the exterior member is developed on a plane in the image memory 104 based on the read shape information (step S304).

When the exterior flag 291 is "2" ("2" in step S305), the exterior image processing unit 112 reads the exterior scan image from the storage unit 105 onto the image memory 104, and from the read exterior scan image, A frame image is generated by developing the outer edge shape of the exterior member on a plane (step S306).

When the design flag 292 is "1" ("1" in step S307), the exterior image processing unit 112 reads the design ID identifying the design image selected by the user from the storage unit 105 (step S308). . Next, the exterior image processing unit 112 reads the design image identified by the read design ID from the storage unit 105 onto the image memory 104 (step S309). The exterior image processing unit 112 determines whether the photo flag 293 is "0" or "1" (step S310). When the photograph flag 293 is "1" ("1" in step S310), the exterior image processing unit 112 applies the base color within the frame image (step S311). When the photo flag 293 is "0" ("≠1" in step S310), the exterior image processing unit 112 does not apply the base color. Next, the image processing unit 106 draws the pattern in the read design image in the frame image (step S312).

When the design flag 292 is "2" ("2" in step S313), the exterior image processing unit 112 reads the design scan image from the storage unit 105 onto the image memory 104 (step S314). The exterior image processing unit 112 determines whether the photo flag 293 is "0" or "1" (step S315). When the photograph flag 293 is "1" ("1" in step S315), the exterior image processing unit 112 applies the base color within the frame image (step S316). When the photo flag 293 is "0" ("≠1" in step S315), the exterior image processing unit 112 does not apply the base color. Next, the image processing unit 106 draws the pattern in the read design scan image in the frame image (step S317).

When the exterior image is generated, the printer control unit 108 causes the printer unit 12 to print the exterior image (step S318).

Thus, the print execution operation is completed.

(8) Photograph Transmission Operation The photograph transmission operation will be described with reference to the flowchart shown in FIG.

Touch panel 310 of mobile terminal 300 receives an instruction to send a photograph from the user (step S341). Next, touch panel 310 receives selection of a photo image from the user (step S342). Next, touch panel 310 accepts a transmission button operation from the user (step S343).

The second wireless communication unit 315 of the mobile terminal 300 and the wireless communication unit 116 of the image forming apparatus 10 authenticate the partner device (step S344).

If the device authentication succeeds, the main control unit 301 reads one photo image from the storage unit 302 (step S345). Next, the second wireless communication unit 315 wirelessly transmits the read photo image, and the wireless communication unit 116 receives the photo image (step S346). Wireless communication unit 116 writes the received photo image to storage unit 105 (step S348).

If reading of the photo image from storage unit 302 has not ended ("NO" in step S347), main control unit 301 returns control to step S345 and repeats the process.

If reading of the photo image from the storage unit 302 has ended (“YES” in step S 347 ), the main control unit 301 transmits an end notification via the second wireless communication unit 315 . The exterior image processing unit 112 receives the end notification via the wireless communication unit 116 (step S349).

As a result, the process of transmitting the photographic image from the portable terminal 300 to the image forming apparatus 10 ends.

1.7 Summary As described above, according to the embodiment, the camera 318 of the mobile terminal 300 captures the surroundings of the image forming apparatus 10 as a subject to generate a photographic image. The generated photo image is transmitted from mobile terminal 300 to image forming apparatus 10 and stored in storage unit 105 . The exterior image processing unit 112 of the image forming apparatus calculates the area of each color in the photographic image stored in the storage unit 105, and sets the color corresponding to the largest area as the base color. When the base color is selected, the base color is applied within the outer edge shape of the exterior member, and a pattern is drawn to generate the exterior image. Next, the printer unit 12 prints the exterior image on the sheet.

The user cuts out the exterior image printed on the sheet along its shape, and attaches the obtained sheet on which the exterior image is printed to the exterior member 32 of the image forming apparatus 10 .

Since the base color represented by the exterior image is the color obtained by photographing the surroundings where the image forming apparatus 10 is arranged, the color of the surface of the exterior member 32 to which the sheet is attached is the color of the surface of the exterior member 32 where the image forming apparatus 10 is arranged. It is in keeping with the colors that surround it.

As a result, there is no need to manufacture an exterior member with a color that matches the surroundings, and the number of parts of the image forming apparatus 10 does not increase. Therefore, the manufacturing cost does not increase.

2 Modification (1)
An image forming system 1a as a modified example (1) of the image forming system 1 of the embodiment will be described.

As shown in FIG. 26A, the image forming system 1 includes an image forming apparatus 10 and a portable terminal 300, which are components of the image forming system 1, and a head-mounted display 500 (glasses-type display device). It is configured.

2.1 Head mounted display 500
The head mounted display 500 is worn on the user's head and used.

As shown in FIG. 27, the head mounted display 500 includes a CPU 501, a ROM 502, a RAM 503, a display control section 504, a display section 505 (display means), an operation section 506 (input means), an image processing section 507 (drawing means), a camera 508 (photographing means), a camera input section 509, a wireless communication section 513 (second communication means), and the like. A wireless communication unit 513 includes an antenna 510 , a wireless communication circuit 511 , and a wireless communication control unit 512 .

(1) CPU 501, ROM 502 and RAM 503
ROM 502 stores a plurality of control programs. Each control program is a computer program configured by combining a plurality of instruction codes indicating instructions to the CPU 501 in order to achieve a predetermined function.

A RAM 503 provides a work area and the like for the CPU 501 .

The CPU 501 operates according to the control program stored in the ROM 502, so that the head mounted display 500 achieves some of its functions. The CPU 501, ROM 502 and RAM 503 functionally constitute a main control section 521 and an information storage section 522, as shown in FIG.

(2) Information storage unit 522
The information storage unit 522 has an area for storing device information, base color, and the like.

(3) Camera 508, camera input unit 509 and image processing unit 507
Camera 508 generates a photographic image by taking a picture. Camera 508 outputs the generated photographic image to image processing section 507 via camera input section 509 .

When the user uses the head mounted display 500, the user sets the direction of the head mounted display 500 so that the imaging range of the camera 508 includes the image forming apparatus 10 arranged in the area. be.

A camera input unit 509 relays photographic images between the camera 508 and the image processing unit 507 .

The image processing unit 507 (rendering means) receives a photographic image from the camera 508 via the camera input unit 509 . The image processing unit 507 also receives device information and identification information indicating the base color from the main control unit 521 . Here, the image processing unit 507 may receive identification information indicating the basic tone pattern in addition to the identification information indicating the basic tone color. That is, the image processing unit 507 may receive identification information indicating the design.

The image processing unit 507 also identifies an image portion representing the device identified by the received device information in the photographic image. Next, the image processing unit 507 identifies a position corresponding to the outer edge shape included in the received device information in the identified image portion. Next, the image processing unit 507 overwrites the received base color in the shape corresponding to the outer edge shape at the specified position in the photographic image. Here, the image processing unit 507 may overwrite the base tone pattern in addition to the base color. In other words, the image processing unit 507 may overwrite the design consisting of the base color and the base pattern.

Next, the image processing unit 507 outputs the photographic image overwritten with the base color to the display unit 505 via the display control unit 504 .

When the operation unit 506 receives (described later) that the base color (or base pattern) is not approved, the image processing unit 507 changes the base color (or base pattern) to another base color (or base pattern). , and the changed base color (or base pattern) is overwritten in the shape corresponding to the outer edge shape in the photographic image.

(4) Antenna 510, Wireless Communication Circuit 511, and Wireless Communication Control Unit 512
An antenna 510, a wireless communication circuit 511, and a wireless communication control unit 512 perform wireless communication based on wireless LAN standards such as IEEE802.11a, b, n, and ac.

Antenna 510 wirelessly transmits and receives radio signals to and from image forming apparatus 10 .

The radio communication circuit 511 performs frequency selection, frequency conversion, and the like of radio signals transmitted and received by the antenna 510 .

The wireless communication control section 512 demodulates the signal received from the wireless communication circuit 511 and outputs it to the main control section 521 . Also, the wireless communication control unit 512 modulates the signal received from the main control unit 521 and outputs the modulated signal to the wireless communication circuit 511 .

(5) Operation unit 506
An operation unit 506 is composed of a plurality of buttons. When the user presses these buttons, an operation signal corresponding to each button is generated. Next, the generated operation signal is output to the main control section 521 .

Pressing one button out of the plurality of buttons indicates the start of operation on head mounted display 500 . When this button is pressed, the operation unit 506 generates an operation start signal indicating the start of operation on the head mounted display 500 and outputs the generated operation start signal to the main control unit 521 .

Pressing another button indicates that the design (color, pattern, or combination of color and pattern) represented in the photographic image displayed by the display unit 505 is approved.

Further pressing of another button indicates that the design shown in the photographic image displayed by the display unit 505 is not approved.

(6) Display control unit 504 and display unit 505
The display control unit 504 receives the photographic image from the image processing unit 507 and outputs the received photographic image to the display unit 505 .

The display unit 505 is, for example, a liquid crystal display. The display unit 505 displays the photographic image received from the display control unit 504 .

An example of the screen displayed by the display unit 505 is shown in FIG. 26(b). The image displayed on screen 531 shown in this figure includes a partial image representing the image forming apparatus. Originally, the screen 531 shown in FIG. 26B includes images of the background of the image forming apparatus, that is, images of walls, ceilings, and office furniture arranged around them. For the sake of simplification, illustration of images such as the background of the image forming apparatus is omitted in this figure.

(7) Main control section 521
The main control unit 521 receives an operation signal indicating a user's operation from the operation unit 506 . Upon receiving the operation signal, the main control unit 521 performs processing according to the operation signal.

When receiving an operation start signal from the operation unit 506 indicating the start of operation of the head mounted display 500 by the user as an operation signal, the main control unit 521 controls the image processing unit 507, the camera input unit 509, and the display control unit 504. is controlled so that the photographic image generated by the camera 508 is processed or is output to the display unit 505 as it is for display.

2.2 Operations of Image Forming System 1a Operations of the image forming system 1a will be described with reference to flowcharts shown in FIGS.

The operation unit 506 accepts pressing of a button indicating the start of operation (display) on the head mounted display 500 (step S401).

The wireless communication unit 513 of the head mounted display 500 and the wireless communication unit 116 of the image forming apparatus 10 authenticate the partner device (step S402).

When the device authentication is successful, the main control unit 521 generates a display request (step S403), and sends the generated display request to the exterior image processing unit 112 via the wireless communication unit 513 and the wireless communication unit 116. Send. The exterior image processing unit 112 receives the display request (step S404).

Exterior image processing unit 112 reads device information (including outer edge shape) of itself (image forming apparatus 10) from storage unit 105 (step S405), and transmits the read device information to wireless communication unit 116 and wireless communication unit 513. to the main control unit 521 via. The main controller 521 receives the device information (step S406). The main control unit 521 writes the received device information to the information storage unit 522 (step S407).

In addition, the exterior image processing unit 112 reads identification information indicating the base color (or base pattern) from the storage unit 105 (step S408), and transmits the read identification information via the wireless communication unit 116 and the wireless communication unit 513. and transmits it to the main control unit 521 . The main controller 521 receives the identification information (step S409). The main control unit 521 writes the received identification information to the information storage unit 522 (step S410).

Camera 508 generates a photographic image by taking a picture. The photographic image includes a partial image representing the image forming apparatus 10 and a background image on which the image forming apparatus 10 is arranged (step S411).

Next, the image processing unit 507 identifies an image portion in which the device identified by the received device information is represented in the photographic image, and in the identified image portion, the outer edge shape included in the received device information. A corresponding position is identified (step S412).

Next, the image processing unit 507 overwrites (applies) the received base color in the shape corresponding to the outer edge shape in the photograph image (step S413).

Next, the image processing unit 507 outputs the photographic image overwritten with the base color to the display unit 505 via the display control unit 504, and the display unit 505 displays the photographic image (step S414). .

The operation unit 506 receives an input from the user as to whether or not to approve the color represented in the photographic image displayed by the display unit 505 (step S415).

If an input indicating approval is received from the user ("YES" in step S416), the main control unit 521 transmits the approval (approval signal) and the approved base color (or base pattern). The identification information to be identified is transmitted to the exterior image processing section 112 via the wireless communication section 513 and the wireless communication section 116 . The exterior image processing unit 112 receives the approval and the identification information identifying the approved base color (or base pattern) (step S419).

Upon receiving the approval, the exterior image processing unit 112 writes identification information for identifying the approved basic tone color (or basic tone pattern) in the storage unit 105 (step S420). Next, printing is executed using the basic tone color (or basic tone pattern) written in the storage unit (step S421). Here, the print execution in step S421 is the same as the print execution shown in step S111 of FIG.

If an input indicating disapproval is received from the user ("NO" in step S416), the main control unit 521 replaces the received base color (or base pattern) with another base color (or other base pattern). (step S417). Next, the image processing unit 507 applies the changed basic tone color (or other basic tone pattern) to the photographic image (step S418). Next, the main control unit 521 shifts control to step S414 and repeats the process.

This completes the description of the operation of the image forming system 1a.

2.3 Summary As described above, according to the modified example (1), before printing by the image forming apparatus, the user can display an image to be printed on a sheet to be attached to the exterior member of the image forming apparatus using the head-mounted sheet. It can be visually recognized by the display.

In addition, since the wall and ceiling in the area where the image forming apparatus is installed are also displayed on the head-mounted display, the user can see the design applied to the wall and ceiling of the area and the image forming apparatus. It is possible to visually confirm whether or not the design to be printed on the attached sheet is in harmony.

3 Other Modifications Although the present invention has been described based on the above embodiments, it is not limited to the above embodiments. You may make it show below.

(1) In the above embodiments, the camera provided in the mobile terminal photographs the design applied to the walls and ceiling in the area where the image forming apparatus is arranged, and generates a photographic image. However, it is not limited to this.

Instead of the portable terminal, a camera (photographer, photographing device) may be used to photograph the design applied to the wall or ceiling in the area where the image forming device is arranged, and generate a photographic image. A user attaches a recording medium to the camera. A camera consists of a lens that collects light from a subject and forms an image on an image pickup device, an image pickup device (photographing means) such as a CCD that converts the light emitted through the lens into an electrical signal, and a digital and a writing means (output means) for writing the generated photographic image onto a recording medium. A user removes the recording medium in which the photographic image is written from the camera, and the removed recording medium is attached to the image forming apparatus 10 . The image forming apparatus 10 includes reading means (design obtaining means) for reading a photographic image from a recording medium. The reading means reads the photographic image from the recording medium. Thereby, the design drawn in the area where the object (for example, the image forming apparatus 10) should be arranged is obtained.

In addition, the image forming apparatus includes a camera (capturing means) composed of a lens, an image sensor, and an image processing engine. The design may be photographed to generate a photographic image.

(2) In the above embodiment, the exterior image processing unit 112 selects one color or one pattern according to the area occupied by the color or pattern in the photographic image. For example, the exterior image processing unit 112 of the image forming apparatus calculates the area of each color (or pattern) in the photographic image stored in the storage unit 105, and selects the color (or pattern) that occupies the largest area. It is selected as the base color (or base pattern). However, it is not limited to this method.

A photographic image may include multiple colors (or multiple patterns). The exterior image processing unit 112 selects a predetermined number of colors (or patterns) represented in one or more photographic images, from the highest occupied area, for example, three colors. Candidates (or pattern candidates) may be extracted. The exterior image processing unit 112 causes the operation panel 19 to display a predetermined number of color candidates (or pattern candidates). The operation panel 19 accepts designation of one color candidate (or pattern candidate) from the predetermined number of displayed color candidates (or pattern candidates) by user's operation. The extraction unit 112a of the exterior image processing unit 112 extracts the color candidate (or pattern candidate) designated by the user as one base color (or base pattern).

The above method may be applied in variant (1). That is, image forming apparatus 10 transmits a predetermined number of color candidates (or pattern candidates) to head mounted display 500 . The display unit 505 of the head mounted display 500 draws and displays one color candidate (or pattern candidate) out of a predetermined number of color candidates (or pattern candidates) in a shape corresponding to the outer edge shape, Head mounted display 500 asks the user for approval. When the user's approval is obtained, the information indicating the approval and the identification information for identifying the approved color candidate (or pattern candidate) are transmitted to the image forming apparatus 10 . If the user's approval is not obtained, the head-mounted display 500 displays the next color candidate (or pattern candidate) and asks the user for approval. Thereafter, the request for display and approval is repeated for all the predetermined number of color candidates (or pattern candidates). If the user's approval is not obtained for all of the predetermined number of color candidates (or pattern candidates), the fact is transmitted to the image forming apparatus 10 . In this case, the image forming apparatus 10 stops printing the sheet.

Here, the head-mounted display 500 may combine one color candidate and one pattern candidate (that is, as a design) and display them. That is, it is also possible to draw in a shape corresponding to the outer edge shape, apply one color candidate color, and draw the pattern candidate pattern thereon. In this way, all combinations of color candidates and pattern candidates may be displayed in order and the user's approval may be requested.

(3) As described above, the exterior image processing unit 112 extracts one pattern among the plurality of patterns represented in the photographic image. Here, the operation panel 19 may accept designation of one color by user's operation.

The image processing unit 106 generates a frame image (shape) representing the exterior member on a plane on the image memory 104 based on the shape information (outer edge shape and dimensions). Next, the image processing unit 106 applies one designated color within the frame of the frame image developed on the plane, and draws the extracted pattern on the applied color. may be

Here, the storage unit 105 may store a plurality of color samples each representing a plurality of colors. Operation panel 19 may display a plurality of color samples. Further, the operation panel 19 may accept designation of one color sample from the user.

Also, a color specified by the user is represented on the surface of the document, and the image reader unit 11 may read the surface of the document and generate read image data representing the color specified by the user. . The image processing unit 106 generates a frame image (shape) representing the exterior member on a plane on the image memory 104 based on the shape information (outer edge shape and dimensions). Next, the image processing unit 106 applies the color represented by the read image data within the frame of the frame image developed on the plane, and applies the extracted pattern on the applied color. You can also draw.

(4) As described above, the exterior image processing unit 112 extracts one color from the plurality of colors represented in the photographic image. Here, the operation panel 19 may accept designation of one pattern by user's operation.

The image processing unit 106 generates a frame image (shape) representing the exterior member on a plane on the image memory 104 based on the shape information (outer edge shape and dimensions). Next, the image processing unit 106 applies one of the extracted colors in the frame of the frame image developed on the plane, and draws the specified pattern on the applied color. may be

Here, the storage unit 105 may store a plurality of pattern samples each representing a plurality of patterns. The operation panel 19 may display a plurality of pattern samples. Further, the operation panel 19 may accept designation of one pattern sample from the user.

A pattern specified by the user is displayed on the surface of the document, and the image reader unit 11 may read the surface of the document and generate read image data representing the pattern specified by the user. . The image processing unit 106 generates a frame image (shape) representing the exterior member on a plane on the image memory 104 based on the shape information (outer edge shape and dimensions). Next, the image processing unit 106 applies the extracted color within the frame of the frame image developed on the plane, and draws the pattern represented by the read image data on the applied color. , may be

(5) In the above embodiment, one design image is selected from pre-stored design images, or a design is scanned by the image reader unit 11 from a sheet on which the design is applied, and selected. Or use a scanned design. However, it is not limited to this.

The image forming apparatus 10 may include an image drawing unit that generates a design image by drawing using line drawings, characters, symbols, etc., according to a user's operation.

(6) In the paper feeding unit 13 of the image forming apparatus 10, there is a dedicated sheet punched with continuous small holes (perforation lines, perforations) along the outer edge of the exterior image so that it can be easily cut off. It may be accommodated. By printing on this dedicated sheet, it is possible to eliminate troublesome cutting work after printing.

Further, the paper feeding unit 13 of the image forming apparatus 10 may accommodate a special sheet such as a chromic material that develops color in reaction to heat, pressure, or the like. When the use of the special sheet is notified to the image forming apparatus 10 by the user, the fixing unit applies heat and pressure to the special sheet at a temperature and pressure suitable for the special sheet when the special sheet passes through the fixing nip. pressure. As a result, it is possible to make the exterior more rich in design.

(7) In the above embodiment, the image forming apparatus 10 performs image formation by electrophotography, but the present invention is not limited to this. It may be a printing device that performs printing by an inkjet method.

(8) Of the external surface of the object, the size of the outer edge shape of the flat portion (including the portion that becomes flat when unfolded) is the size of the sheet accommodated in the paper feeding section 13 of the image forming apparatus 10. If it is greater than, you can do the following:

The image forming apparatus 10 may be connected, for example, via a LAN to a large-sized image forming apparatus that prints large-sized sheets (for example, A2 size). The image forming apparatus 10 may transmit the exterior image to a large-sized image forming apparatus for printing.

In addition, the image forming apparatus 10 divides the exterior image into two parts vertically, two parts horizontally, or four parts vertically and horizontally according to the size of the sheets accommodated in the sheet feeding unit 13. The divided image data generated by the method may be printed on a sheet.

(9) The object to which the design is applied may be provided with a flat portion or a flat portion by unfolding on the surface of the exterior member of the object. For example, it may be an image forming device, a smart phone, a personal computer, a fixed telephone, or the like. Also, it may be office furniture or the like, or may be a coffee pot, a mug, or the like.

(10) As described above, the image forming apparatus is a computer system with a microprocessor and memory. The memory stores computer programs, and the microprocessor may operate according to the computer programs.

Here, the computer program is constructed by combining a plurality of instruction codes indicating instructions to the computer in order to achieve a predetermined function.

Also, the computer program may be recorded on a computer-readable recording medium such as a flexible disk, hard disk, optical disk, or semiconductor memory.

Also, the computer program may be transmitted via a wired or wireless telecommunication line, a network represented by the Internet, data broadcasting, or the like.

(11) The above embodiments and modifications may be combined.

The printing apparatus according to the present invention has the excellent effect that the exterior of the object to be designed can incorporate the design drawn in the area where the object is arranged, and the design can be printed. It is useful as a technique for printing a design on a sheet because it is attached to an object to be applied.

Reference Signs List 1, 1a image forming system 10 image forming apparatus 11 image reader section 12 printer section 13 paper feeding section 19 operation panel 100 main control section 101 CPU
102 ROMs
103 RAM
104 image memory 105 storage unit 106 image processing unit 107 network communication unit 108 printer control unit 109 scanner control unit 110 input/output unit 111 display control unit 112 exterior image processing unit 112a extraction unit 112b generation unit 113 wireless communication control unit 114 wireless communication circuit 115 antenna 116 wireless communication unit 300 portable terminal 301 main control unit 302 storage unit 303 first communication circuit 304 antenna 305 first communication control unit 306 voice processing unit 307 speaker 308 microphone 309 input/output control unit 310 touch panel 311 input unit 312 antenna 313 Second communication circuit 314 Second communication control unit 315 Second wireless communication unit 316 First wireless communication unit 317 Camera input unit 318 Camera 500 Head mounted display 501 CPU
502 ROMs
503 RAM
504 display control unit 505 display unit 506 operation unit 507 image processing unit 508 camera 509 camera input unit 510 antenna 511 wireless communication circuit 512 wireless communication control unit 513 wireless communication unit 521 main control unit 522 information storage unit

Claims (16)

A printing device for printing a design on a sheet for pasting onto an object to which the design is to be applied,
storage means for storing an outer edge shape of a planar portion or a portion that becomes a planar surface by unfolding, of the exterior surface of the object;
a design acquiring means for acquiring a design drawn on the interior of the area where the object is to be arranged;
generating means for reading the outer edge shape from the storage means, developing the read outer edge shape on a plane, and drawing the acquired design within the frame of the developed shape to generate an exterior image;
a printing means for printing the exterior image on a sheet;
with
The object is the printing device,
The design acquisition means includes:
a photographing means for generating a photographic image by photographing ;
When the printing device is installed inside the area, specifying one design drawn on the interior of the area where the printing device, which is the target object, is installed from the photographic image generated by the photographing means, and extracting means for extracting . The extraction means extracts one pattern or one color from the photographic image as a design,
2. The printing apparatus according to claim 1 , wherein the generation unit draws the extracted pattern or color within a frame of the shape developed on a plane. 3. The printing apparatus according to claim 2 , wherein said extracting means extracts said one pattern or said one color occupying the largest area in said photographic image. In the photographic image, a plurality of patterns or a plurality of colors are represented as a design,
The extracting means extracts a predetermined number of pattern candidates or a predetermined number of color candidates from among the plurality of patterns or colors represented in the photographic image in descending order of occupying a larger area,
Further, input means for receiving designation of one pattern or one color from a predetermined number of pattern candidates or a predetermined number of color candidates by a user's operation,
3. The printing apparatus according to claim 2 , wherein said extraction means extracts said one pattern or said one color specified by said input means. The extraction means extracts one pattern,
Furthermore, an input means for accepting designation of one color by user's operation,
The generating means applies the color specified by the input means within the frame of the shape developed on the plane, and draws the pattern extracted by the extracting means on the applied color. 3. The printing apparatus according to claim 2 , wherein: further comprising storage means for storing a plurality of color samples respectively indicating a plurality of colors;
6. The printing apparatus according to claim 5 , wherein said input means accepts designation of one color from among a plurality of color samples by a user's operation. The extraction means extracts one pattern,
Furthermore, an image reading means for reading the surface to be read is provided,
When a document showing a color is placed on the reading surface, the generation means converts the color shown in the read image read by the image reading means into the frame of the shape developed on the plane. 3. The printing apparatus according to claim 2 , wherein the extracted pattern is drawn on the applied color by the extracting means. The extraction means extracts one color,
Furthermore, an input means for receiving designation of one pattern by user's operation is provided,
The generating means applies the color extracted by the extracting means within the frame of the shape developed on the plane, and draws the pattern specified by the input means on the applied color. 3. The printing apparatus according to claim 2 , wherein: further comprising storage means for storing a plurality of pattern samples each showing a plurality of patterns,
9. The printing apparatus according to claim 8 , wherein said input means accepts designation of one pattern from among a plurality of pattern samples by a user's operation. The extraction means extracts one color,
Furthermore, an image reading means for reading the surface to be read is provided,
The generating means applies the color extracted by the extracting means to the frame of the shape developed on the plane, and the original having the pattern displayed on the applied color is placed on the reading surface. 3. The printing apparatus according to claim 2 , wherein the pattern represented in the read image read by the image reading means is drawn when the pattern is read. Furthermore, an input means for accepting designation of the outer edge shape stored in the storage means by user's operation,
2. The printing apparatus according to claim 1, wherein the generation unit develops the specified outer edge shape on a plane. 2. An exterior surface of the printing device, which is the object, is provided with a concave portion for pasting a sheet on which the exterior image is printed on a portion corresponding to the outer edge shape. The printing device according to . Furthermore, a first communication means for transmitting the acquired outer edge shape and design to a glasses-type display device that is worn on the user's head and performs shooting and display,
The first communication means includes: a second communication means for receiving the outer edge shape and the design; a photographing means for generating a photographic image by photographing; drawing means for overwriting the received design within a frame of a shape corresponding to the received outer edge shape in the image portion where the object is represented in the photographic image; and the photographic image overwritten with the design. display means for displaying; and input means for receiving input from a user as to whether or not to approve the design represented in the photographic image displayed by the display means, and the design is approved by the input means. receiving a signal from the spectacles-type display device that transmits an approval signal indicating approval of the design by the second communication means when the input to that effect is received;
2. The printing apparatus according to claim 1, wherein when the received signal is the approval signal, the generation means generates the exterior image. When the input means receives an input to the effect that the design is not approved, the drawing means changes the design to another design and overwrites the changed design on the photographic image, and the display means: The photographic image overwritten with the changed design is displayed, and the input means further receives an input from the user as to whether or not to approve the design represented in the photographic image displayed by the display means. When an input to the effect that the design is approved is received by the input means, the second communication means transmits an approval signal indicating approval of the design and identification information for identifying the changed design. from the device, the first means of communication receives a signal;
When the received signal includes the approval signal and the identification information, the generation means changes the design acquired by the design acquisition means into a design identified by the received identification information, 14. The printing device according to claim 13 , wherein the design is drawn within the frame of the unfolded shape. A control method for controlling a printing device that prints a design on a sheet for pasting onto an object to which the design is to be applied, comprising:
The printing device comprises storage means for storing an outer edge shape of a planar portion or a portion that becomes planar by unfolding, of the exterior surface of the object,
The control method is
a design acquisition step of acquiring a design drawn on the interior of the area where the object is to be arranged;
a generation step of reading the outer edge shape from the storage means, developing the read outer edge shape on a plane, and drawing the acquired design within the frame of the developed shape to generate an exterior image;
a printing step of printing the exterior image onto a sheet ;
The object is the printing device,
The design acquisition step includes:
a photographing step of generating a photographic image by photographing;
When the printing device is installed inside the area, specifying one design drawn on the interior of the area where the printing device, which is the object, is installed from the photographic image generated by the photographing step, including an extraction step to extract
A control method characterized by: A computer program for controlling a printing device that prints a design on a sheet for attachment to an object to which the design is to be applied, comprising:
The printing device comprises storage means for storing an outer edge shape of a planar portion or a portion that becomes planar by unfolding, of the exterior surface of the object,
The printing device, which is a computer,
a design acquisition step of acquiring a design drawn on the interior of the area where the object is to be arranged;
a generation step of reading the outer edge shape from the storage means, developing the read outer edge shape on a plane, and drawing the acquired design within the frame of the developed shape to generate an exterior image;
a printing step of printing the exterior image on a sheet; and
The object is the printing device,
The design acquisition step includes:
a photographing step of generating a photographic image by photographing;
When the printing device is installed inside the area, specifying one design drawn on the interior of the area where the printing device, which is the object, is installed from the photographic image generated by the photographing step, including an extraction step to extract
computer program.

Images