JP7119338B2 - Information processing system, information processing device, information processing method and program - Google Patents

Description

The present invention relates to an information processing system, an information processing device, an information processing method, and a program.

For electronic data such as image data and document data stored in computers, etc., use applications that process the electronic data (e.g., word processing software, spreadsheet software, presentation software, PDF viewer, etc.) It can be displayed on a display device. The user can use the application that displays the electronic data to operate (turn pages, edit, etc.) the displayed electronic data.

In addition, by displaying a book made up of a plurality of sheets of paper on a display device capable of stereoscopic display, and detecting two objects that perform an operation to turn a part of the plurality of sheets of paper, the object that is turned over based on the detection result is detected. A display device that displays a page on a display unit has been proposed (see Patent Document 1, for example).

However, applications generally differ according to the type of electronic data. Therefore, the user has to learn how to operate each application.

Further, the technique disclosed in Japanese Patent Laid-Open No. 2002-200000 has a problem that it is difficult to perform an operation because an operation is performed on an image of a book or paper that is stereoscopically displayed like AR (Augmented Reality).

Therefore, if electronic data is printed, it is freed from differences in operability for each application such as page turning, writing, deletion, and the like. However, simply printing paper for reading electronic data is not only economical, but also burdens the environment.

SUMMARY OF THE INVENTION It is an object of the present invention to make it possible to confirm data without printing by projecting data onto a medium.

In order to solve the above problems, an information processing system includes a projection control unit that controls projection of an image, a paper detection unit that detects an area of a sheet of paper placed in the projection range of the image, and data consisting of a plurality of pages. a generation unit that generates a projection image to be projected onto the projection range so that an image related to a page selected as a projection target corresponds to the area of the paper , and the generation unit includes the paper detection unit. If the change in the area of the paper detected by matches the condition indicating that the paper has been turned over and the condition indicating that a part of the paper has been lifted and then put back, the The projection image is regenerated so that the image of the page preceding or following the page selected as the projection target corresponds to the area of the paper .

Data can be confirmed without printing by projecting the data onto the medium.

1 is a diagram showing a configuration example of a data manipulation support system 1 according to a first embodiment; FIG. 2 is a diagram illustrating a hardware configuration example of the information processing apparatus 10 according to the first embodiment; FIG. 2 is a diagram showing a hardware configuration example of a paper projection device 20 according to the first embodiment; FIG. 2 is a diagram showing a first example of the appearance of the paper projection device 20 according to the first embodiment; FIG. FIG. 7 is a diagram showing a second example of the appearance of the paper projection device 20 in the first embodiment; 2 is a diagram illustrating functional configuration examples of an information processing apparatus 10 and a paper projection apparatus 20 according to the first embodiment; FIG. FIG. 5 is a diagram showing an example of the arrangement of projection sheets in the projection range; FIG. 4 is a diagram showing a configuration example of a UI area; FIG. 10 is a diagram illustrating an example of layout of a UI area with respect to a paper area; FIG. 4 is a diagram showing an example of movement and rotation of a projection sheet; FIG. 10 is a diagram showing an example of turning over of projection paper; 4 is a flowchart for explaining an example of a processing procedure executed by the information processing apparatus 10 in the first embodiment; 3 is a diagram showing a configuration example of a paper area information storage unit 111; FIG. 7 is a flowchart for explaining an example of a processing procedure of drawing information correspondence processing; 9 is a flowchart for explaining an example of a processing procedure of projection image data update processing; 4 is a flowchart for explaining an example of a processing procedure executed by the paper projection device 20 in the first embodiment; 9 is a flowchart for explaining an example of a processing procedure executed by the information processing apparatus 10 in the second embodiment; FIG. 10 is a diagram showing an example of functional configurations of an information processing apparatus 10 and a paper projection apparatus 20 according to a third embodiment; FIG. 12 is a flowchart for explaining an example of a processing procedure executed by the information processing apparatus 10 in the third embodiment; FIG. FIG. 11 is a flow chart for explaining an example of a processing procedure executed by a paper projection device 20 in the third embodiment; FIG.

A first embodiment will be described below with reference to the drawings. FIG. 1 is a diagram showing a configuration example of a data manipulation support system 1 according to the first embodiment. In FIG. 1 , the data manipulation support system 1 includes an information processing device 10 and a paper projection device 20 . The information processing device 10 and the paper projection device 20 are connected via a cable such as a USB (Universal Serial Bus) cable, or a communication line such as a wireless or wired LAN (Local Area Network). That is, the information processing device 10 may be placed near the paper projection device 20 (for example, on the same desk) or may be placed away from the paper projection device 20 .

The information processing device 10 is a general-purpose computer. In the present embodiment, the information processing apparatus 10 generates an image to be projected by the paper projection device 20 (hereinafter referred to as “projection image data”), and also generates an image arranged in the projection range of the paper projection device 20. A process is executed according to a user's operation on paper (hereinafter referred to as "projection paper").

The paper projection device 20 is a device that projects projection image data onto a projection range and detects a user's operation on a projection paper placed within the projection range.

FIG. 2 is a diagram showing a hardware configuration example of the information processing apparatus 10 according to the first embodiment. The information processing apparatus 10 of FIG. 2 includes a drive device 100, an auxiliary storage device 102, a memory device 103, a CPU 104, an interface device 105, a display device 106, an input device 107, and the like, which are connected to each other via a bus B, respectively.

A program for realizing processing in the information processing apparatus 10 is provided by a recording medium 101 such as a CD-ROM. When the recording medium 101 storing the program is set in the drive device 100 , the program is installed from the recording medium 101 to the auxiliary storage device 102 via the drive device 100 . However, the program does not necessarily need to be installed from the recording medium 101, and may be downloaded from another computer via the network. The auxiliary storage device 102 stores installed programs, as well as necessary files and data.

The memory device 103 reads out and stores the program from the auxiliary storage device 102 when a program activation instruction is received. The CPU 104 implements functions related to the information processing apparatus 10 according to programs stored in the memory device 103 . The interface device 105 is used as an interface for connecting to a network. A display device 106 displays a GUI (Graphical User Interface) or the like by a program. The input device 107 is composed of a keyboard, a mouse, etc., and is used to input various operational instructions.

FIG. 3 is a diagram showing a hardware configuration example of the paper projection device 20 according to the first embodiment. The paper projection device 20 in FIG. 3 has an auxiliary storage device 201, a memory device 202, a CPU 203, an interface device 204, a projector 205, an infrared projector 206, an infrared camera 207, and the like.

A program for realizing processing in the paper projection device 20 is installed in the auxiliary storage device 201 . The memory device 202 reads the program from the auxiliary storage device 201 and stores it when a program activation instruction is received. The CPU 203 executes functions related to the paper projection device 20 according to programs stored in the memory device 202 . The interface device 204 is used as an interface for connecting to the network.

The projector 205 is a device (projector) that projects image data onto a predetermined range (projection range).

Infrared light projector 206 projects infrared light to a projection range. The infrared camera 207 captures an image that visualizes the infrared rays emitted in the projection range.

FIG. 4 is a diagram showing a first example of the appearance of the paper projection device 20 according to the first embodiment. 4, (1) shows a perspective view, (2) shows a top view, (3) shows a left side view, (4) shows a front view, and (5) shows a right side view. , (6) shows a rear view.

As shown in FIG. 4, the infrared light projector 206 is arranged so as to be able to project infrared rays over the entire projection range A1 of the projector 205, and the infrared camera 207 is arranged so as to be able to capture an image of the entire projection range A1. be done. The projection paper is arranged within the projection range A1. In FIG. 4, the projection range A1 is drawn smaller than the size of the paper projection device 20 for the sake of convenience, but in reality, a larger range can be set as the projection range A1.

FIG. 4 shows an example in which the projector 205 and the infrared projector 206 are exposed. A cover may be placed against it.

FIG. 5 is a diagram showing a second example of the appearance of the paper projection device 20 according to the first embodiment. FIG. 5 shows an example in which covers c1 and c2 are installed. The cover c1 may be made of any material as long as it allows infrared rays to pass through. Moreover, the cover c2 has a window w1 for avoiding the projection by the projector 205 from being blocked.

FIG. 6 is a diagram showing a functional configuration example of the information processing device 10 and the paper projection device 20 according to the first embodiment. 6, the information processing apparatus 10 includes a selection unit 11, a conversion unit 12, a switching unit 13, a generation unit 14, an image transmission unit 15, a paper information reception unit 16, a drawing processing unit 17, and the like. Each of these units is implemented by one or more programs installed in the information processing apparatus 10 causing the CPU 104 to execute the process. The information processing apparatus 10 also uses the paper area information storage unit 111 . Each of these storage units can be implemented using, for example, the memory device 103, the auxiliary storage device 102, or a storage device connectable to the information processing device 10 via a network.

The selection unit 11 receives selection of a file to be projected (hereinafter referred to as a “projection target file”) such as a PDF file. The conversion unit 12 generates page image data, which is image data (for example, JPEG data, etc.) for each page of the projection target file.

The switching unit 13 switches a page to be projected (hereinafter referred to as a “projection target page”) among a plurality of page image data.

The generation unit 14 performs projection based on information indicating the position of the projection paper in the projection range A1 by the paper projection device 20, the rotation angle of the projection paper, the size of the projection paper, and the like (hereinafter referred to as "paper area information"). Projection image data is generated so that the page image data of the projection target page is projected onto the area of the projection paper (hereinafter referred to as "paper area") within the range A1.

FIG. 7 is a diagram showing an example of arrangement of projection sheets in the projection range. As shown in FIG. 7, the projection paper p1 is arranged so as to be included in the projection range A1. In FIG. 7, the area formed by the projection paper p1 is the paper area. A plurality of projection sheets p1, p2, . . . , pn may be arranged in the projection range A1. In this case, the position and rotation angle of each projection paper p1 may be different.

The paper area information may be, for example, information including information indicating the projection range A1 and information indicating the paper area in a predetermined coordinate system. The information indicating the projection range A1 may be, for example, a group of coordinate values of each vertex or two diagonal vertices of the projection range A1 in the coordinate system. The information indicating the paper area may be, for example, information including a group of coordinate values of each vertex or two diagonal vertices of the paper area and a rotation angle in the coordinate system.

Projection image data is generated, for example, as an image corresponding to the entire projection range A1. In this case, it is difficult for the user to recognize that the projection target page is applied to the area corresponding to the paper area in the image, and the area other than the paper area (hereinafter referred to as "background area") is projected. (solid black or dark blue solid color), or a form indicating a background area (a pattern such as a geometric pattern, a grain pattern of a wooden desk, etc.). Alternatively, if ON/OFF control of the light source of the paper projection device 20 is possible, the light source may be turned off for the background area. Furthermore, when the projection mechanism (projection lens system, etc.) of the paper projection device 20 has a function of following the object and controlling the projection, projection is not performed on the background area, but only on the paper area. You may make it control a projection mechanism so that it may perform. Further, the configuration may be such that the ON/OFF control of the light source and the control of the projection mechanism such as the projection angle are combined. However, the projection image data may be image data corresponding only to the projection paper p1 and its vicinity instead of image data corresponding to the entire projection range A1, or the page image data to be projected may be used as the projection image data as it is. may be

Note that the generation unit 14 generates projection image data so as to include a UI area for receiving changes such as changes in drawing colors during the drawing mode. A drawing mode is a processing state (period) in which drawing on a projection sheet is valid. For example, it is started by touching the drawing paper with a pen or the like. The pen may be a dedicated pen or a normal pen. Hereinafter, a pen used for drawing will be referred to as a "drawing pen". The drawing pen is desirably a stylus pen or the like that does not actually draw on the drawing target (the projection paper p1 in this embodiment), but a ballpoint pen or pencil is also preferred. For example, it may be possible to draw on the projection paper p1.

FIG. 8 is a diagram showing a configuration example of the UI area. A UI area u1 shown in FIG. 8 includes virtual buttons (represented by images) such as buttons b1 to b8. The button b1 is a button for accepting an instruction to return the projection target page to the previous page. The button b2 is a button for receiving an instruction to change the drawing pen type (hereinafter referred to as "drawing type") to a black ballpoint pen or a black pen. The button b3 is a button for receiving an instruction to change the drawing type to a red ballpoint pen or a red pen. The button b4 is a button for receiving an instruction to change the drawing type to a blue ballpoint pen or a blue pen. The button b5 is a button for accepting an instruction to change the drawing type to a yellow highlighter pen. The button b6 is a button for receiving an instruction to change the drawing type to the eraser. That is, the selection states of the buttons b2 to b6 are exclusive, and the selected button is enlarged. Therefore, FIG. 8 shows a state in which the button b3 is selected. The button b7 is a button for accepting an instruction to save and end projection. The button b8 is a button for accepting an instruction to move the projection target page forward by one page. Note that the drawing types shown in FIG. 8 are merely examples. Other drawing types may be selectable.

The UI area u1 is arranged in the paper area as shown in FIG. 9, for example. FIG. 9 is a diagram illustrating an arrangement example of the UI area with respect to the paper area. In FIG. 9, rectangles indicated by (1) and (2) respectively indicate paper areas. (1) shows an arrangement example of the UI area u1 when the projection paper p1 is placed vertically in the projection range A1. (2) shows an arrangement example of the UI area u1 when the projection paper p1 is placed horizontally in the projection range A1. Note that the UI area u1 may be arranged in an area other than the paper area within the projection range A1.

Return to FIG. The image transmission unit 15 transmits projection image data to the paper projection device 20 .

The paper information receiving unit 16 receives (acquires) paper area information from the paper projection device 20 . The paper area information received by the paper information receiving section 16 is stored in the paper area information storage section 111 . The paper information receiving unit 16 also receives from the paper projection device 20 information indicating that the projection paper p1 has been turned over or the corner of the projection paper p1 has been folded (hereinafter referred to as “operation information”). The operation information received by the paper information receiving section 16 is notified to the switching section 13 .

The drawing processing unit 17 receives from the paper projection device 20 information indicating the drawing content of the drawing pen on the projection paper p1 (hereinafter referred to as “drawing information”). Note that when the drawing pen is a pen that cannot actually draw, the drawing information is information indicating the trajectory of the pen tip of the drawing pen in contact with the projection paper p1. The drawing processing unit 17 updates the page image data to be projected based on the received drawing information. That is, the drawing processing unit 17 reflects the drawing content indicated by the drawing information in the page image data. At this time, the rendering content is reflected according to the selected rendering type.

On the other hand, the paper projection device 20 has an image reception section 21, a projection control section 22, a paper detection section 23, a drawing detection section 24, and the like. These units are implemented by one or more programs installed in the paper projection device 20 causing the CPU 203 of the paper projection device 20 to execute.

The image receiving unit 21 receives projection image data from the information processing device 10 . The projection control unit 22 controls the projection of the projection image data received by the image receiving unit 21 onto the projection range A1 by the projector 205 .

The paper detection unit 23 detects the paper area such as the position, rotation angle, size, etc. for each projection paper p1 in the projection range A1 based on the infrared contrast indicated by the image input from the infrared camera 207 . For example, the paper detection unit 23 detects an area in the image in which the pixel value is equal to or greater than a threshold value as a paper area. The paper detector 23 also detects changes in the paper area. For example, the paper detection unit 23 detects movement of the projection paper p1, rotation of the projection paper p1, removal or addition of the projection paper p1, and the like. The paper detection unit 23 transmits paper area information indicating the paper area to the information processing apparatus 10 when the paper area is detected for the first time after the paper projection device 20 is activated or when a change in the paper area is detected.

FIG. 10 is a diagram showing an example of movement and rotation of the projection paper. In FIG. 10, (1) shows an example of movement of the projection paper p1 within the projection range A1. (2) shows an example of rotation of the projection paper p1. As a result of the rotation, the projection paper p1 may have an inclination with respect to the projection range A1, as shown in (3). For example, this inclination is the rotation angle of the projection paper p1.

Note that when a plurality of sheets of projection paper p1 are arranged within the projection range A1, the paper detection unit 23 detects the paper area of each projection paper p1 and changes in each paper area. Therefore, for example, in a meeting, conference, etc., by placing the projection paper p1 in front of each attendee, the page image data can be projected in front of each attendee. In this case, the paper projection device 20 may be placed at a high place in order to expand the projection range A1.

Return to FIG. The paper detection unit 23 also detects an operation that meets a predetermined condition among the user's operations on the projection paper p1 (for example, turning over, folding a corner, etc.).

FIG. 11 is a diagram showing an example of how the projection paper is turned. As shown in FIG. 11, turning over the projection paper p1 refers to, for example, the operation of lifting the right side or the left side of the projection paper p1 so as to turn it over and then returning the lifted side to its original position. Therefore, turning is different from turning inside out. However, the turning may be replaced by turning inside out.

For example, when the user performs an operation to turn over the projection paper p1, the paper area returns to its original size and shape after the size and shape of the paper area change temporarily. The paper detection unit 23 detects the turning of the projection paper p1 based on the detection of the change in the paper area that satisfies such conditions. The paper detection unit 23 transmits to the information processing apparatus 10 operation information indicating the details of the operation performed on the projection paper p1. Note that turning the projection paper p1 corresponds to an instruction to change the projection target page.

Also, when the user temporarily folds one of the corners of the projection paper p1, one of the four corners of the paper area changes to a shaved shape and then returns to its original shape. The paper detection unit 23 detects corner folds of the projection paper p1 based on detection of such a phenomenon. Note that when the corner fold of the projection paper p1 is detected, the information processing apparatus 10 determines that an instruction to insert a bookmark (or an instruction to add a tag) to the projection target page has been given.

The drawing detection unit 24 detects drawing (trajectory of the pen tip) by the drawing pen on the projection paper p1 included in the projection range A1.

Various techniques can be used as a drawing detection method. For example, a method of detecting the trajectory of the pen tip using a drawing pen equipped with a small camera that captures an image of the pattern printed on the projection paper p1 may be adopted. Alternatively, a method of detecting the trajectory of the pen tip on the projection paper p1 by imaging with a camera other than the drawing pen may be adopted. Alternatively, a method of detecting the drawing content by a projection paper p1 (panel) that detects drawing by a pressure-sensitive method, an electrostatic method, or the like may be employed. In addition, a method is adopted in which the infrared light projector 206 projects infrared rays in a grid pattern to the projection range A1, and the infrared camera 207 detects the drawing by detecting the grid passed by the tip of the drawing pen. good too. Further, when the drawing pen actually draws on the projection paper p1, a method of detecting the actually drawn content as an image may be used. The drawing detection unit 24 transmits to the information processing apparatus 10 drawing information including a coordinate group indicating the detected drawing content (the trajectory of the pen tip). Alternatively, image data in which drawing content is reflected in the projection target image data or the page image data that is part of the projection target image data may be transmitted to the information processing apparatus 10 as the drawing information.

Processing procedures executed by the information processing apparatus 10 and the paper projection apparatus 20 will be described below. FIG. 12 is a flowchart for explaining an example of a processing procedure executed by the information processing apparatus 10 in the first embodiment.

For example, when the paper information receiving unit 16 first detects a paper area in the paper projection device 20, when a change in the paper area is detected due to movement or rotation of the projection paper p1, or when the paper area increases or decreases. When the paper area information transmitted from the paper projection device 20 is received at this time (when the projection paper p1 increases or decreases) (Yes in S101), the paper information receiving unit 16 stores the received paper area information in the paper area information storage unit 111. (S102).

FIG. 13 is a diagram showing a configuration example of the paper area information storage unit 111. As shown in FIG. As shown in FIG. 13, each record in the paper area information storage unit 111 includes items such as "projection target file name", "coordinate information", "rotation angle", "projection target page", and "drawing mode". include. "Projection target file name" is the file name of the projection target file. "Coordinate information" is information indicating the position and range of the paper area. The “coordinate information” is, for example, the coordinate values of the diagonal vertices of the paper area. "Rotation angle" is information indicating the rotation angle of the paper area. “Projection target page” is the page number of the page that is currently being projected in the projection target file. The drawing mode is information indicating whether the drawing mode is in progress (TRUE) or not (FALSE). Among these items, "coordinate information" and "rotation angle" are included in the paper area information. Therefore, the paper area information storage unit 111 is configured to be able to store the projection target age, the drawing mode, etc. in association with the paper area information. Note that the paper area information storage unit 111 stores a record for each piece of paper area information.

For example, when a paper area is first detected, if a plurality of projection papers p1p2, . is received and stored in the paper area information storage unit 111 .

Also, when the paper area of any projection paper p1 changes (moves or rotates), the paper area information before the change (before movement or rotation) and the paper area information after the change (after movement or rotation) are received. In this case, the paper information receiving unit 16 updates the paper area information before movement or rotation among the paper area information stored in the paper area information storage unit 111 to the paper area information after movement or rotation. .

Further, when any projection paper p1 is removed from the projection range A1, information indicating that the projection paper p1 has been removed is received in addition to the paper area information in the state before removal of the projection paper p1. . In this case, the paper information receiving unit 16 stores the paper area information stored in the paper area information storage unit 111 and information associated with the paper area information, which will be described later, into the paper area information storage unit 111. delete from

Also, when a new projection paper p1 is added to the projection range A1, the paper area information after the projection paper p1 is arranged is received. In this case, the paper information receiving section 16 stores the paper area information in the paper area information storage section 111 .

When synchronizing the projection target page, drawing mode, etc. for all the projection papers p1, it is not necessary to distinguish between the projection papers p1. Paper area information may be received each time. In this case, in the paper area information storage unit 111, all paper area information may be rewritten each time.

Alternatively, when the selection unit 11 receives the selection of the projection target file (Yes in S103), the conversion unit 12 reads the projection target file into the memory device 103, and the projection target file in the memory device 103 (hereinafter referred to as "projection target file"). file m1") is generated for each page (S104). For example, if the projection target file m1 is a PDF file containing 5 pages, 5 JPEG data are generated as page image data. The generated page image data is stored in the auxiliary storage device 102, for example. Subsequently, the switching unit 13 selects the first page in the projection target file m1 as the projection target page (S105). Specifically, the switching unit 13 associates information indicating the projection target page (for example, “1”) with each piece of paper area information stored in the paper area information storage unit 111 and stores the information in the paper area information storage unit 111 . .

Alternatively, when the paper information receiving unit 16 receives the operation information from the paper projection device 20 (Yes in S106), the paper information receiving unit 16 receives the operation information (hereinafter referred to as “target operation information”) from the projection paper p1. It is determined whether the information indicates the turning of the projection paper p1 or the information indicating the folding of the corner of the projection paper p1 (S107).

If the target operation information is information indicating turning of the projection paper p1 (Yes in S107), the switching unit 13 changes the projection target page according to the operation content indicated by the target operation information (S108). In this case, the target operation information includes paper area information of the projection paper p1 to be turned over as identification information of the projection paper p1. Therefore, for example, when the target operation information indicates an operation of lifting the right side of the projection paper p1 and turning it to the left, the switching unit 13 selects the projection target page associated with the target operation information in the paper area information storage unit 111. 1 is added (that is, the next page is set as the projection target page). On the other hand, when the target operation information indicates an operation of lifting the left side of the projection paper p1 and turning it to the right, the switching unit 13 selects 1 from the projection target page associated with the target operation information in the paper area information storage unit 111. Subtract (that is, use the previous page as the projection target page). However, if the page to be projected is already the last page, the page is not changed to the next page, and if the page to be projected is the first page, the page is not changed to the previous page. However, if the page to be projected is already the last page and switching to the next page is requested, the first page may be set as the page to be projected, or the page to be projected is the first page. In some cases, when switching to the previous page is requested, the final page may be the projection target page. The right side and the left side of the projection paper p1 are based on the direction of the page image projected on the projection paper p1. Specifically, based on a state in which the vertical direction of the page image matches the user (that is, a state in which the upward direction of the page image is away from the user and the downward direction is close to the user), the projection paper p1 A right side and a left side are determined. However, the direction of turning may not be distinguished. For example, the next page or the previous page may be the projection target page regardless of which direction the projection paper p1 is turned.

Alternatively, when the operation information indicating turning is received for the first time, the switching unit 13 associates the target operation information with the target operation information in the paper area information storage unit 111 regardless of the direction of turning included in the target operation information. One may be added to the projection target page. In this case, the switching unit 13 stores an operation related to the operation information as an operation to advance the page, and stores an operation opposite to the operation as an operation to return the page. For example, if the operation information received for the first time indicates an operation of lifting the right side of the projection paper p1 and turning it over to the left, the operation is stored as an operation for advancing the page, and the operation of lifting the left side and turning it over to the right (turning over) is stored. is stored as a page-back operation. The switching unit 13 switches pages based on the stored contents.

Alternatively, for example, when the page image data is a horizontally long image and the projection paper p1 is placed in the horizontal direction, and the page image data faces the user, the upper or lower side of the projection paper p1 is turned over. Thus, the turning of the projection paper p1 may be detected. In this case, the switching unit 13 may advance the projection target page when the lower side is turned over, and return the projection target page when the upper side is turned over.

Note that, in the present embodiment, when a plurality of previous projection sheets p1 are arranged within the projection range A1, an example in which the respective projection contents are different (independent) will be described. are placed within the projection range A1, and if you want to project the same projection target page on all the projection paper p1, that is, if you want to project synchronously, the projection target page is set for each paper area information Instead, only one needs to be stored. Alternatively, in step S108, all projection target pages stored for each piece of paper area information in the paper area information storage unit 111 may be updated.

On the other hand, if the target operation information is information indicating that the corner of the projection paper p1 has been folded (No in S107), the paper information receiving unit 16 gives (adds) bookmark information to the projection target file ( S109). That is, the paper information receiving unit 16 refers to the projection target page stored in the paper area information storage unit 111 in association with the paper area information included in the target operation information, and selects, among the pages of the projection target file, Bookmark information is given (added) to the page corresponding to the projection target page (S109). That is, as will be described later, in this case, the target operation information includes the paper area information of the projection paper p1 whose corner is folded.

Alternatively, when the drawing processing unit 17 receives drawing information (hereinafter referred to as “target drawing information”) from the paper projection device 20 (Yes in S110), the drawing processing unit 17 executes drawing information handling processing (S111). ). In the drawing information corresponding processing, processing corresponding to target drawing information is executed. Details of the drawing information correspondence processing will be described later.

Following steps S102, S105, S107 and S111, the information processing apparatus 10 executes projection image data update processing (S112). After execution of step S112, steps after step S101 are repeated. Details of the projection image data update process will be described later.

Next, details of step S111 will be described. FIG. 14 is a flowchart for explaining an example of the processing procedure of the drawing information correspondence processing.

In step S201, the drawing processing unit 17 determines whether the drawing mode corresponding to the paper area information included in the target drawing information (that is, the paper area information corresponding to the projection paper p1 as the drawing target) is TRUE. determine whether The drawing mode is a parameter indicating whether drawing on the projection paper p1 is valid or not, and is stored in the paper region information storage unit 111 in association with each piece of paper region information. Regarding the drawing mode, "TRUE" indicates that drawing is enabled, and "FALSE" indicates that drawing is disabled. For example, the drawing mode can be set to "TRUE" by touching the projection paper p1 with the drawing pen. Note that when the drawing modes of all the projection papers p1, p2, .

If the drawing mode stored in the paper area information storage unit 111 associated with the paper area information included in the target drawing information is "FALSE" (No in S201), the drawing processing unit 17 The drawing mode associated with the information and stored in the paper area information storage unit 111 is set to "TRUE" (S202). That is, the fact that the drawing information is received for the projection paper p1 whose drawing mode is "FALSE" means that the drawing pen touches the projection paper p1. Therefore, in this case, the drawing mode corresponding to the projection paper p1 is updated to "TRUE".

On the other hand, if the drawing mode stored in the paper area information storage unit 111 associated with the paper area information included in the target drawing information is "TRUE" (Yes in S201), the drawing processing unit 17 It is determined whether or not the position is included in the UI area u1 (S203). Determination of whether or not the drawing position is included in the UI area u1 is made by combining coordinate information indicating the drawing position (contact position of the drawing pen) included in the target drawing information and coordinate information indicating the range of the UI area u1. It is possible by comparison. Since the UI area u1 is arranged at a predetermined position in the paper area, the coordinate information indicating the range of the UI area u1 can be specified based on the paper area information included in the target drawing information.

If the drawing position is included in the UI area u1 (Yes in S203), the drawing processing unit 17 executes a process corresponding to the button operated in the UI area u1 (FIG. 8) (S204).

For example, when the drawing position (that is, the contact position of the drawing pen) is included in the range of the button b1, the drawing processing unit 17 stores the paper region information storage unit in association with the paper region information included in the target drawing information. 1 is subtracted from the projection target page stored in 111, and the process of step S112 (FIG. 15) is called. However, when the projection target page is "1", nothing may be executed, and the final page may be set as the projection target page. Similarly, when the drawing position is included in the range of the button b8, the drawing processing unit 17 selects the projection target page stored in the paper region information storage unit 111 in association with the paper region information included in the target drawing information. is added by 1, and the process of step S112 (FIG. 15) is called. However, if the projection target page is the last page of the projection target file, nothing may be executed, or the first page may be the projection target page.

If the drawing position is within the range of any of the buttons b2 to b6, the drawing type corresponding to the button related to the drawing position is associated with the paper area information included in the target drawing information. Stored in the storage unit 111 .

Further, when the drawing position is within the range of the button b7, the drawing processing unit 17 generates a file of the same data format as the projection target file based on each page image data of the projection target file m1, The file is saved in the auxiliary storage device 102, for example. At this time, the projection target file may be overwritten and saved, or may be saved as a file separate from the projection target file. After saving the file, the drawing processing unit 17 deletes (clears) the projection target file m1, the page image data, and the contents of the paper area information storage unit 111 . As a result, the projection of the projection target file m1 ends.

On the other hand, if the drawing position is not included in the UI area u1 (No in S203), the drawing processing unit 17 associates the paper area information included in the target drawing information and stores it in the paper area information storage unit 111. The page image data related to the projection target page stored in the paper area information storage unit 111 in association with the paper area information is updated based on the drawing content included in the target drawing information according to the drawing type. (S205). Specifically, the drawing processing unit 17 executes drawing on the page image data according to the trajectory (coordinate group) included in the target drawing information according to the pen type and pen color of the drawing type. . Further, when the eraser is selected as the drawing type, the drawing processing unit 17 deletes the drawing content of the portion corresponding to the trajectory indicated by the target drawing information in the page image data.

Next, details of step S112 will be described. FIG. 15 is a flowchart for explaining an example of a processing procedure of projection image data update processing.

In step S301, the generation unit 14 determines whether page image data has been generated. That is, it is determined whether or not a file to be projected has already been selected and steps S104 and S105 of FIG. 12 have been executed. If page image data has not been generated (No in S301), subsequent processing is not executed. If a paper area is detected by the paper projection device 20 before the projection target file is selected and step S112 is called in response to receiving paper area information indicating the paper area, the determination in step S301 is No. .

If page image data has been generated (Yes in S301), the generation unit 14 determines whether or not one or more pieces of paper area information are stored in the paper area information storage unit 111 (S302). When one or more pieces of paper area information are stored in the paper area information storage unit 111 (Yes in S302), the generation unit 14 performs projection including the page image data of the projection target page on the paper area based on each piece of paper area information. Image data is generated (S303). That is, in the projection image data whose entire area is the projection range A, the page image data of the projection target page associated with each piece of paper area information is superimposed on the paper area indicated by each piece of paper area information. At this time, each page image data is enlarged or reduced and rotated according to the size and rotation angle indicated by each paper area information. Therefore, the page image data for the paper area corresponding to the A4 projection paper p1 is enlarged or reduced so as to fit within the A4 size, and the page image data for the paper area corresponding to the A3 projection paper p1 is changed to the A3 size. scaled up or down to fit. The above description does not mean that the size of the projection paper p1 is limited to A3 or A4, and any size may be selected as the size of the projection paper p1.

Subsequently, the generation unit 14 determines whether or not the paper area information (projection paper p1) associated with the drawing mode of "TRUE" is stored in the paper area information storage unit 111 (S305). If there is no corresponding paper area information (No in S305), the process proceeds to step S307. If there is corresponding paper area information (Yes in S305), the generation unit 14 synthesizes (superimposes) an image indicating the UI area u1 on a predetermined position of the paper area related to the paper area information in the projection image data ( S306). That is, projection image data including the UI area u1 within the paper area is generated.

On the other hand, if no paper area information is stored in the paper area information storage unit 111 (No in S302), the generation unit 14 generates projection image data whose entire surface is the background area (S304). The paper area information is deleted from the paper area information storage unit 111 when the projection paper p1 is removed. Therefore, when all of the projection paper p1 is removed, it appears to the user that the page image data is hidden.

If No in step S305, following step S306 or step S304, the image transmission unit 15 transmits the generated projection image data to the paper projection device 20 (S307).

Next, a processing procedure executed by the paper projection device 20 will be described. FIG. 16 is a flowchart for explaining an example of a processing procedure executed by the paper projection device 20 in the first embodiment.

For example, when the paper detection unit 23 detects a new paper area within the projection range A1, or detects a change in any of the already detected paper areas (Yes in S401), the paper detection unit 23 , the paper area information indicating the paper area newly detected in the projection range A1 or the paper area information corresponding to the changed paper area is transmitted to the information processing apparatus 10 (S402). When multiple paper areas are detected, paper area information for each of the multiple paper areas is transmitted. Further, when the movement or rotation of any paper area is detected, the paper area information before movement or rotation of the paper area and the paper area information after movement or rotation are transmitted to the information processing apparatus 10 . be. Further, when any projection paper p1 is removed from the projection range A1, in addition to the paper area information indicating the paper area before removal of the projection paper p1, the projection paper p1 corresponding to the paper area information is removed. Information indicating that is transmitted to the information processing apparatus 10 .

When synchronizing the projection target page, drawing mode, etc. for all the projection papers p1, it is not necessary to distinguish between the projection papers p1. Paper area information may be transmitted.

Alternatively, when the image receiving unit 21 receives the projection image data transmitted from the information processing apparatus 10 (Yes in S403), the projection control unit 22 causes the projector 205 to project the projection image data onto the projection range A1. (S404). Here, page image data is applied to each paper area in the projected image data. Therefore, the page image data is projected onto the surface of the projection paper p1 arranged in the projection range A1. Further, when the projection paper p1 is moved or rotated, the page image data is superimposed on the projection image data according to the paper area after movement or rotation. Therefore, from the user's point of view, the page image data appears to move or rotate following the projection paper p1. Therefore, for example, the user can move the projection paper p1 closer to himself or rotate the projection paper p1 to display the page image data to himself or another person (a person next to him, a person in front of him, a person in the publication, etc.). can be easily seen.

Alternatively, when the paper detection unit 23 detects that any projection paper p1 has been turned (Yes in S405), the paper detection unit 23 detects the paper area information of the turned projection paper p1 and the turning direction. to the information processing apparatus 10 (S406). The information indicating the turning direction is, for example, information indicating whether the right side of the projection paper p1 was lifted and turned to the left, or whether the left side of the projection paper p1 was lifted and turned to the right.

Alternatively, when the paper detection unit 23 detects that any corner of the projection paper p1 is temporarily folded (Yes in S407), the paper detection unit 23 detects the paper area information of the projection paper p1 and the corner is folded, and the operation information is transmitted to the information processing apparatus 10 (S408).

Alternatively, when the drawing detection unit 24 detects drawing with the drawing pen on the surface of any projection paper p1 (Yes in S409), the drawing detection unit 24 detects the paper area information of the projection paper p1 to be drawn and the drawing. drawing information including the content (trajectory of the tip of the drawing pen) is transmitted to the information processing apparatus 10 (S410). For example, when the drawing pen touches the surface of the paper, information including the paper area information of the projection paper p1 related to the surface of the paper and the coordinates of the touched position becomes the drawing information. Also, when the pen tip of the drawing pen moves on the paper surface, information including the paper area information of the projection paper p1 related to the paper surface and the coordinate group of the movement trajectory becomes the drawing information.

Note that when step S112 is executed after step S105 of FIG. 12 is executed (that is, when step S112 is executed immediately after the projection target file is selected), the information processing apparatus 10 performs In step S303, the projection image data of all pages of the projection target file m1 may be transmitted, and in step S304, the projection image data of all the pages may be transmitted. In this case, the paper projection device 20 may sequentially (continuously) project the projection image data of all pages in step S304 of FIG. 16, and then project the projection image data of the first page. good. By doing so, when the projection target file is printed, it is possible for the user to grasp the overall volume (total number of pages) of the projection target file at the start of projection, which the user can feel due to the thickness of the printed matter. can.

As described above, according to the first embodiment, an image of a page of electronic data to be manipulated (projection target file) is projected onto the projection paper p1. can be viewed. Further, the user can manipulate the electronic data (for example, turning pages, adding bookmarks, etc.) by manipulating the projection paper p1. Further, the user can edit the electronic data by drawing on the projection paper p1 with a drawing pen. Therefore, according to the present embodiment, by projecting data onto a medium, it is possible to check the data without printing.

Next, a second embodiment will be described. 2nd Embodiment demonstrates a different point from 1st Embodiment. Therefore, points that are not particularly mentioned may be the same as those in the first embodiment.

In the second embodiment, an example in which page image data of different files are projected for each of a plurality of projection paper p1 in the projection range A1 (that is, the relationship between the projection target file and the projection paper p1 is 1:1). example) will be described. In the second embodiment, the processing procedure of FIG. 17 is executed instead of that of FIG.

FIG. 17 is a flow chart for explaining an example of a processing procedure executed by the information processing apparatus 10 in the second embodiment. In FIG. 17, the same steps as in FIG. 12 are given the same step numbers, and the description thereof will be omitted.

In the second embodiment, steps S121 and S122 are executed before step S104 if Yes in step S103 (that is, if selection of a file to be projected has been accepted).

In step S121, the conversion unit 12 stores the total number of the existing projection target file m1 and the newly selected projection target file (hereinafter referred to as "the number of projection target files") in the paper area information storage unit 111. (that is, the number of sheets of projection paper p1) or less.

If the number of projection target files is equal to or less than the number of paper area information (Yes in S121), the conversion unit 12 assigns newly selected projection target files to unused paper area information (S122). The unused paper area information is the paper area information to which none of the existing projection target files m1 is assigned (that is, the paper area of the projection paper p1 that is not set as the projection destination of any of the existing projection target files m1). information). Assignment of the projection target file to the paper area information may be performed, for example, by associating the file path name of the projection target file with the paper area information. Subsequently, steps after step S104 are executed.

Further, in the second embodiment, in step S303 of FIG. 15, the generating unit 14 generates a projection target associated with the paper area information for each piece of paper area information stored in the paper area information storage unit 111. The page image data corresponding to the file and the projection target page are superimposed on the projection target image data so as to apply to the paper area indicated by the paper area information in the projection target image data. As a result, projection image data in which page image data of different projection target files are projected onto each projection paper p1 is generated.

Next, a third embodiment will be described. In the third embodiment, differences from the first or second embodiment will be explained. Therefore, points not particularly mentioned may be the same as those in the first or second embodiment.

FIG. 18 is a diagram showing a functional configuration example of the information processing device 10 and the paper projection device 20 according to the third embodiment. In FIG. 18, the same parts as in FIG. 6 are denoted by the same reference numerals, and the description thereof will be omitted. In FIG. 18 , the paper projection device 20 further has a gesture detection section 25 . The gesture detection unit 25 is implemented by processing that causes the CPU 203 of the paper projection device 20 to execute one or more programs installed in the paper projection device 20 .

When the gesture detection unit 25 detects a predetermined gesture (movement such as a gesture or hand movement) by the user, the gesture detection unit 25 transmits information indicating the content of the gesture (hereinafter referred to as “gesture information”) to the information processing apparatus 10 . As a method for detecting gestures, it is possible to employ motion detection using a general motion sensor, image recognition, or the like.

On the other hand, the information processing device 10 further has a gesture processing section 18 . The gesture processing unit 18 is realized by processing that one or more programs installed in the information processing apparatus 10 cause the CPU 104 to execute.

Upon receiving the gesture information, the gesture processing unit 18 executes processing according to the gesture information.

FIG. 19 is a flowchart for explaining an example of a processing procedure executed by the information processing apparatus 10 in the third embodiment. In FIG. 19, the same step numbers as in FIG. 12 are assigned to the same steps, and the description thereof will be omitted. In FIG. 19, steps S131 and S132 are added.

When the gesture processing unit 18 receives the gesture information transmitted from the paper projection device 20 (Yes in S131), the gesture processing unit 18 executes processing according to the gesture indicated by the gesture information (S132).

For example, when one of the projection paper p1 is torn (hereinafter referred to as "gesture A"), gesture information including paper area information of the projection paper p1 and information indicating that the projection paper p1 is torn. is received. In this case, the gesture processing unit 18 deletes the page image data associated with the paper area information in step S132. Therefore, when a save is instructed after that, the projection target file corresponding to the page image data is saved with the page related to the page image data removed. By tearing the projection paper p1, the projection paper p1 is removed from the projection range A1. Therefore, in this case, the paper area information indicating that the projection paper p1 has been removed is received (Yes in S101), and after the paper area information is deleted from the paper area information storage unit 111, step S112 is executed. be done. Therefore, projection image data is generated in which the page image data is not projected onto the projection paper p1.

Further, when the projection paper p1 facing a certain user is moved in the projection range A1 and handed over to the opposite user (hereinafter referred to as "gesture B"), the paper area information of the projection paper p1 and , and information indicating that the projection paper p1 has been handed over. In this case, the gesture processing unit 18 changes the rotation angle of the paper area information by 180 degrees in step S132. Therefore, in this case, in the subsequently executed step S112, projection image data is generated in which the page image data for the projection paper p1 is reversed so as to face the opposite user. As the projection paper p1 moves, the position information of the paper region information stored in the paper region information storage unit 111 regarding the projection paper p1 has already been updated to the state after the movement (S102 ), and in step S132, the position information of the paper area information may not be updated.

Also, when any projection paper p1 is lifted (hereinafter referred to as "gesture C"), paper area information before the projection paper p1 is lifted and information indicating that the projection paper p1 is lifted. Gesture information is received that includes: In this case, in step S132, the gesture processing unit 18 adds flag information indicating that the paper area information stored in the paper area information storage unit 111 has been lifted. As a result, in step S303 of FIG. 15, the generation unit 14 enlarges the page image data corresponding to the paper area information to which the flag information is added so as to protrude from the paper area indicated by the paper area information, and produces a projected image. superimposed on the data. At this time, the degree of enlargement may be changed according to the lifted height or the distance between the lifted projection paper p1 and the lifted user's face (or eyes). In this case, the height, distance, or the like may be included in the gesture information.

Also, when one of the projection paper p1 is folded in half in the long side direction and then opened (returned) (for example, after folding a horizontal A3 paper into A4 size, When opened) (hereinafter referred to as "gesture D"), the original (for example, A3 state) paper area information of the projection paper p1 and the fact that the projection paper p1 is temporarily folded in half. Gesture information is received that includes information indicating and. In this case, in step S132, the gesture processing unit 18 adds information indicating that the paper has been temporarily folded in half to the paper area information stored in the paper area information storage unit 111 . In step S303 of FIG. 15, if there is paper area information to which the flag information is added, the generation unit 14 adds the paper area information to the left half of the paper area indicated by the paper area information in the projection image data. The page image data of the associated projection target page is superimposed, and the page image data of the next page of the projection target page is superimposed on the right half. As a result, two pages are aggregated and projected onto the projection paper p1 corresponding to the paper area information.

Also, when a plurality of projection papers p1 on which different page image data are projected are bundled (for example, after each projection paper p1 is stacked, each projection paper p1 is lifted in a substantially vertical direction, When aligned on a desk or the like where the projection range A1 is formed (hereinafter referred to as "gesture E"), the overlapping order of the paper area information of each of the plurality of projection papers p1 and the plurality of projection papers and information indicating that p1 has been bundled. In this case, in step S132, the gesture processing unit 18 generates new page image data in which the page image data corresponding to each of the plurality of pieces of paper area information included in the gesture information are arranged in pages in the order of overlap included in the gesture information. A file is generated and saved in, for example, the auxiliary storage device 102 .

FIG. 20 is a flowchart for explaining an example of a processing procedure executed by the paper projection device 20 in the third embodiment. In FIG. 20, the same step numbers as in FIG. 16 are assigned to the same steps, and the description thereof is omitted. In FIG. 20, steps S411 and S412 are added.

When the gesture detection unit 25 detects a gesture (user's motion on the projection paper p1) that satisfies any one of the gestures A to E (Yes in S411), the gesture detection unit 25 detects the paper area information of the projection paper p1 targeted for the gesture. and information indicating the content of the gesture (type of gesture A to gesture E) is transmitted to the information processing apparatus 10 (S412).

Note that the third embodiment may be combined with the second embodiment.

Further, in each of the above-described embodiments, the turning or corner folding of the projection paper p1 may also be detected based on gesture detection using a motion sensor or the like instead of the infrared projector 206 and infrared camera 207.

Further, the paper area in the projection range A1 may also be detected using a normal digital camera instead of the infrared projector 206 and the infrared camera 207. FIG.

Further, in each of the above embodiments, the information processing device 10 may be connected to the paper projection device 20 via the Internet or the like. For example, the information processing device 10 may be a computer that constitutes a cloud system. In this case, the selection of the projection target file may be performed using a browser or the like on a terminal connected to the information processing apparatus 10 via the Internet or the like.

Also, the paper projection device 20 may have all the functions of the information processing device 10 in FIG. 6 or 18 . For example, a program that causes the information processing apparatus 10 to function as each part of the information processing apparatus 10 shown in FIG. 6 or FIG. You may allow

Also, the paper projection device 20 does not have the infrared projector 206 and the infrared camera 207, and another device having the infrared projector 206 and the infrared camera 207 is connected to the paper projector 20 via a communication line. may In this case, the paper projection device 20 has the image receiving section 21 and the projection control section 22 shown in FIG. A gesture detection unit 25 may be provided.

In each of the above-described embodiments, an example in which the sheet medium on which the image is projected is paper has been described. Each of the above embodiments may be applied. In this case, a sheet-like medium (for example, a medium that can be placed flat on a desk or the like) is suitable as the medium.

Note that each function of the embodiments described above can be realized by one or a plurality of processing circuits. Here, the "processing circuit" in this specification means a processor programmed by software to perform each function, such as a processor implemented by an electronic circuit, or a processor designed to perform each function described above. devices such as ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays) and conventional circuit modules.

In each of the above embodiments, the data manipulation support system 1 is an example of an information processing system. The drawing processing unit 17 is an example of an updating unit. The gesture detection unit 25 is an example of a motion detection unit. The gesture processing unit 18 is an example of a processing execution unit. The paper information receiving section 16 is an example of an obtaining section. The paper detector 23 is an example of a medium detector.

Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications can be made within the scope of the gist of the present invention described in the claims. Transformation and change are possible.

10 Information processing device 11 Selection unit 12 Conversion unit 13 Switching unit 14 Generation unit 15 Image transmission unit 16 Paper information reception unit 17 Drawing processing unit 18 Gesture processing unit 20 Paper projection device 21 Image reception unit 22 Projection control unit 23 Paper detection unit 24 Drawing detection unit 25 Gesture detection unit 100 Drive device 101 Recording medium 102 Auxiliary storage device 103 Memory device 104 CPU
105 interface device 111 paper area information storage unit 201 auxiliary storage device 202 memory device 203 CPU
204 Interface device 205 Projector 206 Infrared projector 207 Infrared camera B Bus

JP 2016-76234 A

Claims (8)

a projection control unit that controls projection of an image;
a paper detection unit that detects an area of the paper arranged in the projection range of the image;
a generation unit that generates a projection image to be projected onto the projection range so that an image of a page selected as a projection target from data consisting of a plurality of pages corresponds to the area of the paper ;
has
The generation unit detects that the change in the area of the paper detected by the paper detection unit is a condition indicating that the paper has been turned over and that a part of the paper has been lifted and then put back. Regenerate the projection image so that the image of the page preceding or following the page selected as the projection target corresponds to the area of the paper if the indicated condition is met.
An information processing system characterized by: When there is a change in the area of the paper detected by the paper detection unit, the generation unit generates the projection image so as to correspond to the area of the paper after the change.
2. The information processing system according to claim 1, characterized by: When the change in the area of the paper detected by the paper detection unit first matches the condition , the generation unit selects either the previous page or the next page of the page selected as the projection target. or regenerate the projection image so that the image for the page in one direction corresponds to the area of the paper , and then if the condition is met, the direction of the paper turn is the first match. If the direction is the same as the case, the projection image is regenerated so that the image of the page in the one direction corresponds to the area of the paper with respect to the page selected as the projection target, and the turning of the paper is performed. If the direction is opposite to the first match, the image of the page selected as the projection target in the direction opposite to the one direction is projected so as to correspond to the area of the paper . regenerate the image,
3. The information processing system according to claim 1, wherein: a drawing detection unit that detects drawing on the paper ;
an updating unit that updates the data based on the content of the drawing;
4. The information processing system according to any one of claims 1 to 3, characterized by comprising: a motion detection unit that detects a user's motion with respect to the paper ;
a processing execution unit that executes processing on the data according to the motion detected by the motion detection unit;
5. The information processing system according to any one of claims 1 to 4, characterized by comprising: an acquisition unit that acquires information indicating the area of the paper arranged in the projection range of the image;
a generation unit that generates a projection image to be projected onto the projection range so that an image of a page selected as a projection target from data consisting of a plurality of pages corresponds to the area of the paper ;
has
The generation unit indicates that a change in the area of the paper acquired by the acquisition unit is a condition indicating that the paper has been turned over and that a part of the paper has been lifted and then put back. Regenerate the projection image so that the image of the page preceding or following the page selected as the projection target corresponds to the area of the paper if the condition is met.
An information processing device characterized by: an acquisition procedure for acquiring information indicative of the area of the paper positioned within the projection range of the image;
a generation procedure for generating a projection image to be projected onto the projection range such that an image of a page selected as a projection target from data consisting of a plurality of pages corresponds to the area of the paper ;
When the change in the area of the paper obtained by the obtaining procedure meets the condition indicating that the paper has been turned over and the condition indicating that a part of the paper has been lifted and then put back. (2) a regeneration step of regenerating the projection image so that the image of the page preceding or following the page selected as the projection target corresponds to the area of the paper ;
An information processing method characterized in that a computer executes an acquisition procedure for acquiring information indicative of the area of the paper positioned within the projection range of the image;
a generation procedure for generating a projection image to be projected onto the projection range such that an image of a page selected as a projection target from data consisting of a plurality of pages corresponds to the area of the paper ;
When the change in the area of the paper obtained by the obtaining procedure meets the condition indicating that the paper has been turned over and the condition indicating that a part of the paper has been lifted and then put back. (2) a regeneration step of regenerating the projection image so that the image of the page preceding or following the page selected as the projection target corresponds to the area of the paper ;
A program characterized by causing a computer to execute

Images