JP7796372B2 - Information processing system, information processing method, and program - Google Patents

Description

The present disclosure relates to an information processing system, an information processing method, and a program. More specifically, the present disclosure relates to an information processing system, an information processing method, and a program that perform information processing related to image capture at an image capture position within a target space.

The photographing terminal described in Patent Document 1 is an example of a conventional technology for displaying a diagram of a target space and information about the photographing location on a display unit. The photographing terminal displays a first screen that shows the location where the equipment will be installed as a position on a layout diagram. When the first screen is displayed on the photographing terminal and the user selects (e.g., taps) a desired position on the first screen, the camera function of the photographing terminal is activated and a second screen is displayed on the photographing terminal. On the second screen, at least a portion of the construction information is displayed in a changeable manner on the camera function screen (display unit). When the photographer takes a photograph using the camera function, the construction information linked to the position selected on the first screen is attached to the photographed image.

While users of the photography terminal described in Patent Document 1 can determine the location where the device is installed from the first screen, there is a problem in that it is difficult to determine where and what kind of subject they should photograph.

Japanese Patent Application Laid-Open No. 2021-141388

The purpose of this disclosure is to provide an information processing system, information processing method, and program that enable users to easily understand where and what subject they should photograph.

An information processing system according to one aspect of the present disclosure includes a storage unit, a display unit, and an operation unit . The storage unit stores a space map that is a diagram of a target space, shooting positions set on the space map, and object information related to an object to be photographed at the shooting positions. The display unit displays the space map, a marker corresponding to the shooting positions, and the object information. The operation unit accepts a user operation to switch the display mode of the display unit between a simple display mode and a detailed display mode. When the display mode of the display unit is the detailed display mode, in an unphotographed state where photography at the shooting positions has not yet been completed, the display unit displays the space map with the marker superimposed at a first display position corresponding to the shooting position on the space map and the object information superimposed at a second display position corresponding to the first display position . When the display mode of the display unit is the simple display mode, the display unit displays the space map with the marker superimposed at the first display position.

An information processing method according to one aspect of the present disclosure is an information processing method for an information processing system including a storage unit and a display unit, and includes a display step and an operation receiving step . The storage unit stores a space map that is a diagram of a target space, shooting positions set on the space map, and object information related to an object to be photographed at the shooting positions. The display unit displays the space map, a marker corresponding to the shooting positions, and the object information. The operation receiving step receives a user operation for switching the display mode of the display unit to either a simple display mode or a detailed display mode. The display step, when the display mode of the display unit is the detailed display mode , displays the space map with the marker superimposed at a first display position corresponding to the shooting position and the object information superimposed at a second display position corresponding to the first display position in an unphotographed state where photography at the shooting positions has not yet been completed. When the display mode of the display unit is the simple display mode, the space map with the marker superimposed at the first display position is displayed .

One aspect of the program of the present disclosure is a program for causing one or more processors of a computer system to execute the information processing method.

FIG. 1 is a block diagram of an information processing system according to an embodiment. FIG. 2 is a flowchart showing an example of processing in the information processing system. FIG. 3 is an explanatory diagram showing a first example of a screen display of the information processing system. FIG. 4 is an explanatory diagram showing a second example of a screen display of the information processing system. FIG. 5 is an explanatory diagram showing a third example of a screen display of the information processing system.

(Embodiment)
An information processing system 1, an information processing method, and a program according to an embodiment will be described below with reference to the accompanying drawings. However, the embodiment described below is merely one of various embodiments of the present disclosure. The embodiment described below can be modified in various ways depending on the design, etc., as long as the object of the present disclosure can be achieved.

(overview)
The "target space" referred to in this disclosure is a construction site of a building (e.g., an apartment building, a detached house, an office building, a hospital, a factory, a commercial facility, etc.) before, during, or after construction. However, the "target space" is not limited to this example.

During construction work such as the construction of a new building, remodeling, or renovation, a portion of the target space (construction site) of the building being constructed is photographed, and the captured images are used to confirm and inspect the condition of the building and the construction status. Information processing system 1 presents information related to such photography work to workers (users), thereby assisting in the confirmation and inspection of the condition of the building and the construction status. In other words, information processing system 1 assists in the confirmation and inspection associated with the construction of a building.

Information processing system 1 has the function of processing photographed images of a portion of the target space of a building and information related to the photographed images to assist in confirmation and inspection associated with the construction of the building. Users such as workers and inspectors photograph various parts of the target space before, during, or after construction and save the photographed images.

In this embodiment, as an example, the building is an apartment building, and a dwelling unit in the apartment building is being renovated. In this case, the target space is the apartment building (construction site), which is the construction site for the renovation. When renovating a dwelling unit, a current situation survey is first conducted to confirm the condition of the dwelling unit before the renovation. Then, demolition work is carried out to remove unnecessary parts and equipment from the current dwelling unit, and the main renovation work is carried out. The main work includes carpentry work, equipment work, piping work, wiring work, painting work, interior work, etc. Once the main work is completed, a final inspection is carried out to inspect the dwelling unit after the renovation. During each of the current situation survey, demolition work, main work, and completion inspection, photography may be carried out to confirm and inspect the condition of the building and the construction status, etc.

As shown in FIG. 1, the information processing system 1 of this embodiment includes a memory unit 12 and a display unit 14. The memory unit 12 stores a spatial diagram A1, which is a diagram of the target space, a shooting position A2 set on the spatial diagram A1, and target information A31 related to the target to be photographed at the shooting position A2. As shown in FIG. 3, the display unit 14 displays the spatial diagram A1, markers (markers M1 to M4) corresponding to the shooting position A2, and target information A31. In an unphotographed state where photography at the shooting position A2 has not yet been completed, the display unit 14 displays the spatial diagram A1 with markers (markers M1 to M4) superimposed at a first display position corresponding to the shooting position A2 on the spatial diagram A1 and with target information A31 superimposed at a second display position corresponding to the first display position.

Here, the space diagram A1 is a diagram representing the target space. The space diagram A1 may be, for example, a floor plan of the target space, or, if the target space is a dwelling unit, a floor plan of the dwelling unit. The space diagram A1 may be a two-dimensional drawing, a three-dimensional drawing, or a virtual image representing the target space. The shooting position A2 is the position where a shooting target in the target space is photographed using the shooting unit 15 (described later). The target information A31 is information about the shooting target to be photographed at the shooting position A2, and includes at least one of label information (such as a name) representing the shooting position where the shooting target is photographed and label information (such as a name) representing the shooting target. Note that in the example screens shown in Figures 3 to 5, multiple shooting positions A2 are set on the space diagram A1, but the number of shooting positions A2 may be one or more and can be changed as appropriate. In Figures 3 to 5, four markers M1 to M4 representing four shooting positions A2 are displayed, respectively, but the number of markers can be changed as appropriate depending on the number of shooting positions A2. In the following description, the multiple markers M1 to M4 displayed superimposed on the spatial diagram A1 may be collectively referred to as markers Mn (n is a natural number).

The first display position corresponding to the shooting position A2 is, for example, the same position as the shooting position A2, but is not limited to a position that exactly coincides with the shooting position A2 and may be near the shooting position A2. The second display position is a position that corresponds to the first display position, and therefore also a position that corresponds to the shooting position A2. The second display position is, for example, a position near the first display position. The second display position is set, for example, at a position where the marker Mn displayed at the first display position and the target information A31 displayed at the second display position are displayed adjacent to each other.

According to this embodiment, in an uncaptured state where capture at capture position A2 is incomplete, the display unit 14 displays a spatial diagram A1 in which markers Mn (M1-M4) are superimposed at the first display position on the spatial diagram A1 and target information A31 is superimposed at the second display position (see FIG. 3). Therefore, based on the display content of the display unit 14, the user can easily identify the target and the capture position A2 where the target will be captured, and can easily understand where and what target should be captured.

Functions similar to those of the information processing system 1 can also be embodied in an information processing method. The information processing method of this embodiment is an information processing method executed by the information processing system 1 and includes a display step. The memory unit 12 stores a spatial diagram A1, which is a diagram of the target space, a shooting position A2 set on the spatial diagram A1, and target information A31 related to the target to be photographed at the shooting position A2. The display unit 14 displays the spatial diagram A1, a marker Mn corresponding to the shooting position A2, and the target information A31. In the display step, in an unphotographed state where photography at the shooting position A2 has not yet been completed, the spatial diagram A1 is displayed with the marker Mn superimposed at a first display position corresponding to the shooting position A2 in the spatial diagram A1 and the target information A31 superimposed at a second display position corresponding to the first display position.

The information processing method can also be embodied in a program. The program of this embodiment is a program for causing one or more processors of a computer system to execute the information processing method. The program may be recorded on a non-transitory recording medium readable by the computer system.

(detail)
(1) Overall Configuration First, the overall configuration of the information processing system 1 will be described with reference to FIG.

The information processing system 1 is, for example, a tablet terminal, a smartphone, or a dedicated construction photography terminal used to take construction photos.

The information processing system 1 includes a computer system having one or more processors and memory. At least some of the functions of the information processing system 1 are realized by the processor of the computer system executing a program recorded in the memory of the computer system. The program may be recorded in memory, provided via a telecommunications line such as the Internet, or provided recorded on a non-transitory recording medium such as a memory card.

As shown in FIG. 1, the information processing system 1 includes a processing unit 11, a memory unit 12, an operation unit 13, a display unit 14, a photographing unit 15, and a communication unit 16.

(2) Processing Unit The processing unit 11 includes one or more processors of the information processing system 1. The processing unit 11 executes a program to realize a predetermined function.

The processing unit 11 is configured to perform overall control of the information processing system 1. For example, the processing unit 11 controls the display of images by the display unit 14. The processing unit 11 also controls the photographing operation by the photographing unit 15.

The processing unit 11 may also have a function to perform image processing on the captured image A4 generated by the photographing unit 15. The image processing may include, for example, a process of combining an electronic whiteboard (electronic small blackboard) on which construction details are written with the captured image A4.

(3) Storage Unit The storage unit 12 stores information used in the information processing system 1. The storage unit 12 stores, for example, a space diagram A1, shooting positions A2, and display information A3. In this embodiment, a plurality of shooting positions A2 are set on the space diagram A1, and therefore the storage unit 12 stores the plurality of shooting positions A2. The display information A3 includes a plurality of pieces of object information A31 related to a plurality of shooting targets to be photographed at the plurality of shooting positions A2, and a plurality of pieces of order information A32.

The multiple pieces of target information A31 correspond one-to-one to the multiple shooting positions A2. Each piece of target information A31 includes information about the target to be photographed at the corresponding shooting position A2, such as label information for the shooting position A2 at which the target is photographed and/or label information for the target. The label information for the shooting position A2 at which the target is photographed includes, for example, information indicating the location of the shooting position A2. If the target space is a dwelling unit in an apartment building, the label information for the shooting position A2 preferably includes information indicating which room the shooting position A2 is. In FIG. 3, the target information A31 displayed in the second display position includes, as label information for the shooting position A2, information such as whether the shooting position A2 is a Japanese-style room or a bathroom. The label information for the target preferably includes, for example, the name of the target. In FIG. 3, the target information A31 displayed in the second display position includes, as label information for the target, information such as whether the target is a hook-on light or a recessed light. The label information for the target may also be a symbol or the like representing the target.

Furthermore, the multiple pieces of order information A32 correspond one-to-one to the multiple shooting positions A2, and each of the multiple pieces of order information A32 is information that indicates the order in which shooting will be performed at the corresponding shooting position A2. In other words, the memory unit 12 further stores order information A32 that indicates the order in which shooting will be performed at the multiple shooting positions A2.

The memory unit 12 also stores the captured image A4 generated by the imaging unit 15. The memory unit 12 stores the captured image A4 in association with information about the imaging position A2 at which the captured image A4 was captured.

It is preferable that the memory unit 12 has a rewritable non-volatile memory such as an EEPROM (Electrically Erasable Programmable Read Only Memory) or flash memory.

(4) Operation Unit The operation unit 13 is a user interface device that accepts user operations. For example, the operation unit 13 includes at least one of a touch panel, a button, or a key switch that accepts user operations by hand (manual operations), and a microphone that accepts user operations by voice (voice operations).

When the operation unit 13 receives a user operation, it outputs an operation signal corresponding to the user operation to the processing unit 11. The processing unit 11 performs processing corresponding to the operation signal. For example, the processing unit 11 controls the display unit 14 or the imaging unit 15 in response to the operation signal.

If the operation unit 13 has a touch panel that accepts manual operations, user operations on the touch panel include, for example, tap, double tap, long tap, drag, swipe, flick, pinch in, and pinch out.

If the operation unit 13 has a microphone that accepts voice operations, the operation unit 13 performs voice recognition processing on the microphone output to generate an operation signal corresponding to the voice uttered by the user. Note that the processing unit 11 may also perform the voice recognition processing.

(5) Image Capturing Unit The image capturing unit 15 includes a two-dimensional image sensor such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor. The image capturing unit 15 functions as a camera that captures color or monochrome images (still images) and generates data of the captured image A4. The image capturing unit 15 stores the data of the captured image A4 in the storage unit 12.

For example, if the touch panel display realizes the functions of both the operation unit 13 and the display unit 14, when the user taps a marker Mn displayed on the screen of the display unit 14, the image capture unit 15 enters a standby state for capturing an image. Furthermore, when the user performs a predetermined operation on the operation unit 13, the image capture unit 15 captures an image. The memory unit 12 stores the data of the captured image A4 in association with information about the capturing position A2 corresponding to the marker Mn tapped by the user. Note that the display unit 14 displays the target information A31 at the second display position until a capture is performed at the capturing position A2, and displays the captured image A4 at the second display position once a capture is performed at the capturing position A2.

(6) Communication Unit The communication unit 16 includes a communication interface device. The communication unit 16 is capable of communicating with an external system via the communication interface device. In this disclosure, "capable of communication" means being able to send and receive signals directly or indirectly via a network, a repeater, or the like, using an appropriate communication method such as wired communication or wireless communication. Note that wired communication refers to wired communication via, for example, a twisted pair cable, a dedicated communication line, or a LAN (Local Area Network) cable. Wireless communication refers to wireless communication that complies with standards such as Wi-Fi (registered trademark) or low-power wireless (specified low-power wireless) that does not require a license.

The communication unit 16 exchanges data with external systems. For example, the communication unit 16 acquires data for the space diagram A1 from an external system. For example, the communication unit 16 acquires data for the space diagram A1, which is a diagram of the target space, from an external CAD system or external server, and stores the data for the space diagram A1 in the memory unit 12. Note that the space diagram A1 in this embodiment is a floor plan of a dwelling unit, as shown in Figures 3 to 5.

(7) Display Unit (7.1) Overview of the Display Unit The display unit 14 is a display device using an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diode), or the like, and has a screen for displaying images. The display unit 14 is controlled by the processing unit 11.

In addition, the touch panel display may perform the functions of both the operation unit 13 and the display unit 14.

When photographing a subject at one or more photographing positions A2 set in the target space, the display unit 14 displays a space diagram A1, which is a diagram of the target space, as shown in Figures 3 to 5.

Here, the screen of the display unit 14 is provided with at least a first display area 141 that displays the spatial diagram A1. In the example screens of Figures 3 to 5, the screen of the display unit 14 is further provided with a fourth display area 144 that displays a slide button SB1 for switching the display mode of the display unit 14 between simple display mode and detailed display mode.

Figures 3 and 4 are example screens when the display mode of the display unit 14 is the detailed display mode, and Figure 5 is an example screen when the display mode of the display unit 14 is the simple display mode. Here, the simple display mode is a mode in which the display unit 14 displays a spatial diagram A1 with a marker Mn superimposed at a first display position corresponding to the shooting position A2. The detailed display mode is a mode in which the display unit 14 displays a spatial diagram A1 with a marker Mn superimposed at the first display position and with target information A31 or a photographed image A4 superimposed at a second display position corresponding to the first display position.

The fourth display area 144 displays a slide button SB1 for switching the detailed display mode ON or OFF. Turning the detailed display mode ON means setting the display mode to the detailed display mode. Turning the detailed display mode OFF means setting the display mode to the simple display mode. Here, when the user slides the slide button SB1 to the ON position, the operation unit 13 accepts the user's operation to set the display mode to the detailed display mode. Also, when the user slides the slide button SB1 to the OFF position, the operation unit 13 accepts the user's operation to set the display mode to the simple display mode. In other words, the information processing system 1 further includes an operation unit 13 that accepts the user's operation to switch the display mode of the display unit 14 between the simple display mode and the detailed display mode.

Below, we will explain the display screen of the display unit 14 when the display mode is detailed display mode and the display screen of the display unit 14 when the display mode is simple display mode.

(7.2) Display Screen in Detail Display Mode The display screen when the display mode of the display unit 14 is the detail display mode will be described with reference to FIGS. 3 and 4. FIG.

Figure 3 is an example of a display screen when shooting has not been completed at all of the multiple shooting positions A2, and Figure 4 is an example of a display screen when shooting has been completed at all of the multiple shooting positions A2.

First, we will explain the display screen when shooting has not been completed at all of the multiple shooting positions A2, with reference to Figure 3.

The display unit 14 displays a space diagram A1, which is a diagram of the target space. The screen of the display unit 14 is provided with a first display area 141 for displaying the space diagram A1.

The display unit 14 displays a spatial diagram A1 in which multiple markers M1-M4 and multiple pieces of target information A31 are superimposed. The multiple markers M1-M4 correspond one-to-one to the multiple shooting positions A2. Each of the multiple markers M1-M4 is displayed at a first display position, which is the same position as the corresponding shooting position A2, and represents the location of the corresponding shooting position A2. The multiple pieces of target information A31 also correspond one-to-one to the multiple shooting positions A2. Each piece of target information A31 is displayed at a second display position corresponding to the first display position of the marker Mn representing the corresponding shooting position A2, and is displayed near the marker Mn representing the corresponding shooting position A2. In other words, the screen of the display unit 14 has a rectangular second display area 142 for displaying the target information A31 near the marker Mn. This second display area 142 displays characters representing the target information A31 (e.g., the name of the shooting position A2, the name of the subject being photographed, etc.).

Here, the second display area 142 displaying the target information A31 is smaller than the first display area 141 displaying the spatial diagram A1, and the second display area 142 is larger than the display size of the marker Mn. Because the second display area 142 is larger than the display size of the marker Mn, the amount of target information displayed in the second display area 142 can be increased. Furthermore, because the second display area 142 is smaller than the first display area 141, it is easier to see which of the multiple displayed markers M1 to M4 the target information A31 corresponds to, which has the advantage of allowing the user to easily grasp the shooting position A2 of the shooting target represented by the target information A31.

In addition, order information A32 is displayed superimposed on each of the markers M1 to M4. In this embodiment, order information A32 is represented by consecutive numbers such as 1, 2, 3, etc.

For example, at the first display position corresponding to shooting position A2 where the first photograph is taken, a marker M1 and the number [1] as sequence information A32 are displayed superimposed on the spatial diagram A1, and at the second display position corresponding to the first display position, target information A31 is displayed superimposed on the spatial diagram A1. Similarly, at the first display position corresponding to shooting position A2 where the nth photograph is taken (n is a natural number), a marker Mn and the number [n] as sequence information A32 are displayed superimposed on the spatial diagram A1, and at the second display position corresponding to the first display position, target information A31 is displayed superimposed on the spatial diagram A1.

In this way, when the display mode of the display unit 14 is the detailed display mode, the display unit 14 displays a spatial diagram A1 in which the marker Mn is superimposed at the first display position and the target information A31 is superimposed at the second display position in an unphotographed state. Therefore, based on the display on the display unit 14, the user can easily understand where and what kind of target they should photograph.

The display unit 14 also displays sequence information A32 in association with a marker Mn at each of the multiple first display positions. Therefore, the user can easily understand the sequence in which to take photographs at the multiple photographing positions A2 based on the display on the display unit 14.

In this embodiment, as shown in FIG. 3 etc., the display unit 14 displays the marker Mn and the sequence information A32 (numbers) so that they overlap each other. However, the sequence information A32 may be displayed near the marker Mn. In other words, the position where the marker Mn is displayed on the screen of the display unit 14 may be different from the position where the sequence information A32 is displayed.

Next, we will explain an example of a screen when photography is taken at all multiple photography positions A2, with reference to Figure 4.

The display unit 14 displays a spatial diagram A1, which is a diagram of the target space. The display unit 14 displays the spatial diagram A1 in which multiple markers M1-M4 and multiple captured images A4 are superimposed. The multiple markers M1-M4 correspond one-to-one to the multiple shooting positions A2. Each of the multiple markers M1-M4 is displayed at a first display position, which is the same position as the corresponding shooting position A2, and represents the location of the corresponding shooting position A2. The multiple captured images A4 also correspond one-to-one to the multiple shooting positions A2. Each of the multiple captured images A4 is displayed at a second display position corresponding to the first display position of the corresponding marker Mn, and is displayed near the corresponding marker Mn. Here, the screen of the display unit 14 has a rectangular third display area 143 near the marker Mn for displaying the captured image A4. This third display area 143 displays the captured image A4 of the target taken at the corresponding shooting position A2. In this way, when the display mode of the display unit 14 is the detailed display mode, in a photographed state in which a photographed image A4 photographed at the photographing position A2 exists, the display unit 14 displays the space diagram A1 in which the marker Mn is superimposed at the first display position and the photographed image A4 is superimposed at the second display position. Therefore, since the photographed image A4 is displayed at the second display position, the user can easily understand that the photographing was performed at the photographing position A2 of the corresponding marker Mn.

Here, the third display area 143 displaying the captured image A4 is smaller than the first display area 141 displaying the spatial diagram A1, and the third display area 143 is larger than the display size of the marker Mn. Because the third display area 143 is larger than the display size of the marker Mn, it is possible to display a larger size of the captured image A4. Furthermore, because the third display area 143 is smaller than the first display area 141, it has the advantage of making it easier to see which of the multiple markers Mn the captured image A4 corresponds to.

Note that Figure 4 shows an example screen when shooting has been completed at all of the multiple shooting positions A2, but if shooting has been completed at some of the multiple shooting positions A2, the multiple shooting positions A2 will include unshooting positions where shooting has not yet been completed and shooting positions where shooting has been completed.

In this case, the display unit 14 displays a marker Mn at the first display position corresponding to the unphotographed position on the spatial diagram A1, and displays target information A31 related to the target to be photographed at the unphotographed position at the second display position corresponding to the first display position. Furthermore, the display unit 14 displays a marker Mn at the first display position corresponding to the photographed position on the spatial diagram A1, and displays a photographed image A4 photographed at the photographed position at the second display position corresponding to the first display position. This allows the user to check the photographed image A4 at the photographed position among the multiple photographing positions A2, and to check target information A31 of the target to be photographed at the unphotographed position.

(7.3) Display Screen in Simple Display Mode The display screen when the display mode of the display unit 14 is the simple display mode will be described with reference to FIG.

The display unit 14 displays a space diagram A1, which is a diagram of the target space. The screen of the display unit 14 is provided with a first display area 141 for displaying the space diagram A1.

The display unit 14 displays a spatial diagram A1 in which multiple markers M1 to M4 are superimposed. The multiple markers M1 to M4 correspond one-to-one to the multiple shooting positions A2. Each of the multiple markers M1 to M4 is displayed at a first display position, which is the position of the corresponding shooting position A2, and represents the corresponding shooting position A2.

As described above, when the display mode of the display unit 14 is the simplified display mode, the display unit 14 displays a spatial diagram A1 with markers M1-M4 superimposed at the first display position. When the user sets the display mode of the display unit 14 to the simplified display mode, the display unit 14 displays a spatial diagram A1 with only marker Mn representing the shooting position A2 superimposed, which has the advantage of simplifying the display on the display unit 14. In other words, in the simplified display mode, the target information is not displayed at the second display position corresponding to the first display position on the spatial diagram A1. Note that when the display mode of the display unit 14 is the detailed display mode, the display unit 14 displays a spatial diagram A1 with markers M1-M4 superimposed at the first display position, and with target information A31 superimposed at the second display position in the unphotographed state and with the photographed image A4 superimposed at the second display position in the photographed state. This allows the user to easily identify the subject to be photographed and the shooting position A2 at which to photograph that subject based on the target information A31 displayed at the second display position in the unphotographed state. Furthermore, in the already-photographed state, the user can easily check the photographed image A4 photographed at the photographing position A2 based on the photographed image A4 displayed at the second display position.

(8) Processing Flow Next, the processing flow of the information processing system 1 in the work of taking photos for construction confirmation will be described with reference to Fig. 2. Note that the flowchart shown in Fig. 2 is merely an example, and the order of each step may be changed as appropriate, and steps may be added or omitted as appropriate.

First, input information such as a spatial diagram A1, multiple shooting positions A2, and multiple pieces of display information A3 is input to the information processing system 1 (step ST1). The spatial diagram A1, multiple shooting positions A2, and multiple pieces of display information A3 are input to the communication unit 16 from, for example, an external system. Alternatively, the multiple shooting positions A2 and multiple pieces of display information A3 are input, for example, by a user operating the operation unit 13.

The memory unit 12 stores the information entered in step ST1.

When the necessary information is input in step ST1, the information processing system 1 starts the process of displaying the spatial diagram A1 on the display unit 14.

The processing unit 11 determines whether the display mode is detailed display mode or simple display mode based on the operation information received by the operation unit 13 (step ST2).

If the result of the determination in step ST2 is that the display mode is the detailed display mode (step ST2: Yes), the processing unit 11 sets one of the multiple shooting positions A2 as the shooting position A2 to be determined, and determines whether the shooting position A2 to be determined is a position where an image has already been taken (step ST3). The processing unit 11 sets the multiple shooting positions A2 as the shooting position A2 to be determined in order, for example, according to the shooting order represented by the order information A32, and determines whether the shooting position A2 to be determined is a position where an image has already been taken.

If photography at the photography position A2 to be determined has not been completed, the processing unit 11 determines that the photography position A2 to be determined is not a photography-completed position (step ST3: No) and places the target information A31 superimposed on the second display position corresponding to the photography position A2 to be determined in the spatial diagram A1 (step ST5).

If photography has been completed at the photography position A2 to be determined, the processing unit 11 determines that the photography position A2 to be determined is a photography-completed position (step ST3: Yes) and superimposes the photographed image A4 at the second display position corresponding to the photography position A2 to be determined in the spatial diagram A1 (step ST4).

Next, the processing unit 11 determines whether or not the processing of steps ST3 to ST5 has been performed on all shooting positions A2 in the spatial diagram A1 as the shooting positions A2 to be judged (step ST6).

If there is a shooting position A2 that is not being judged (step ST6: No), the processing unit 11 sets the next shooting position A2 in the shooting order as the shooting position A2 to be judged, returns to step ST3 and repeats the above processing.

If all shooting positions A2 are to be judged (step ST6: Yes), the processing unit 11 places a marker Mn superimposed on the first display position in the spatial diagram A1 (step ST7).

Then, the processing unit 11 performs a display process to display on the display unit 14 a spatial diagram A1 in which the marker Mn is superimposed at the first display position and the target information A31 or the captured image A4 is superimposed at the second display position (step ST8).

On the other hand, if the result of the determination in step ST2 is that the display mode is the simple display mode (step ST2: No), the processing unit 11 places the marker Mn superimposed on the first display position in the spatial diagram A1 (step ST7).The processing unit 11 then performs display processing to display the spatial diagram A1 with the marker Mn superimposed on the first display position on the display unit 14 (step ST8).

When the display unit 14 displays the spatial diagram A1 and the user taps the marker Mn (step ST9: Yes), the operation unit 13 accepts the operation information, and the processing unit 11 activates the photographing unit 15. When the operation unit 13 subsequently accepts a predetermined operation, the photographing unit 15 photographs the subject (step ST10), and the photographed image A4 taken by the photographing unit 15 is stored in the memory unit 12. The processing unit 11 then returns to step ST2 and repeats the processing from step ST2 onwards.

As a result, when a photograph is taken by the photographing unit 15 at any of the photographing positions A2, the photographed image A4 is displayed at the second display position corresponding to the photographing position A2 at which the photograph was taken.

The information processing system 1 repeats the processing of steps ST2 to ST10 until the user operates the operation unit 13 to end the display on the display unit 14, and performs information processing regarding the image captured at the shooting position A2.

(Modification of the embodiment)
Modifications of the embodiment are listed below. The following modifications may be implemented in appropriate combination.

The shape of the marker Mn is not limited to a pin shape and may be any shape.

The order information A32 is not limited to numbers, but may be, for example, letters such as the alphabet, or figures. The order in which the photographs are taken may also be indicated by the color of the marker Mn. In other words, the order information A32 may be indicated by the color of the marker Mn.

In addition, the display unit 14 may display a marker Mn representing a shooting position A2 where image shooting has not yet been completed and a marker Mn representing a shooting position A2 where image shooting has been completed in different display modes (e.g., color, shape, etc.).

The entity that executes the information processing system 1 or information processing method in this disclosure includes a computer system. The computer system is primarily composed of a processor and memory as hardware. At least a portion of the functions of the entity that executes the information processing system 1 or information processing method in this disclosure are realized by the processor executing a program recorded in the computer system's memory. The program may be pre-recorded in the computer system's memory, provided via a telecommunications line, or provided recorded on a non-transitory recording medium readable by the computer system, such as a memory card, optical disc, or hard disk drive. The processor of the computer system is composed of one or more electronic circuits, including a semiconductor integrated circuit (IC) or a large-scale integrated circuit (LSI). The integrated circuits, such as ICs and LSIs, are referred to by different names depending on the degree of integration, and include integrated circuits called system LSIs, very large-scale integration (VLSI), or ultra-large-scale integration (ULSI). Furthermore, a field-programmable gate array (FPGA), which is programmed after the LSI is manufactured, or a logic device capable of reconfiguring the connections within the LSI or the circuit partitions within the LSI, can also be employed as a processor. Multiple electronic circuits may be integrated into a single chip or distributed across multiple chips. Multiple chips may be integrated into a single device or distributed across multiple devices. The computer system referred to here includes a microcontroller having one or more processors and one or more memories. Therefore, the microcontroller is also composed of one or more electronic circuits, including a semiconductor integrated circuit or a large-scale integrated circuit.

Furthermore, it is not essential for the information processing system 1 that multiple functions are consolidated into a single device, and multiple components of the information processing system 1 may be distributed across multiple devices. For example, a device including the imaging unit 15 may be provided separately from a device including the display unit 14.

At least some of the functions of the information processing system 1, for example, the image processing functions in the processing unit 11, may be realized by a server or the cloud (cloud computing), etc.

(summary)
As described above, the information processing system (1) of the first aspect includes a storage unit (12) and a display unit (14). The storage unit (12) stores a spatial diagram (A1) that is a diagram of a target space, shooting positions (A2) set on the spatial diagram (A1), and target information (A31) related to a target to be photographed at the shooting position (A2). The display unit (14) displays the spatial diagram (A1), markers (Mn, M1 to M4) corresponding to the shooting positions (A2), and the target information (A31). In an unphotographed state in which photography at the shooting position (A2) is not yet completed, the display unit (14) displays the spatial diagram (A1) with the markers (Mn, M1 to M4) superimposed at a first display position corresponding to the shooting position (A2) on the spatial diagram (A1) and the target information (A31) superimposed at a second display position corresponding to the first display position.

This allows the user to easily understand where and what subject they should photograph.

In the information processing system (1) of the second aspect, the second display area (142) displaying the target information (A31) is smaller than the first display area (141) displaying the spatial diagram (A1) in the first aspect. The second display area (142) is larger than the display size of the markers (Mn, M1 to M4).

According to this aspect, by making the second display area (142) larger than the display size of the markers (Mn, M1 to M4), the amount of information that can be displayed in the second display area (142) can be increased.

In the information processing system (1) of the third aspect, in the first or second aspect, in a photographed state in which a photographed image (A4) taken at the photographing position (A2) exists, the display unit (14) displays a spatial diagram (A1) in which markers (Mn, M1 to M4) are superimposed at the first display position and the photographed image (A4) is superimposed at the second display position.

According to this aspect, when the image has been captured, the captured image (A4) is displayed at the second display position, allowing the user to check the captured image (A4) captured at the capturing position (A2).

In the information processing system (1) of the fourth aspect, the third display area (143) displaying the captured image (A4) is smaller than the first display area (141) displaying the spatial diagram (A1) in the third aspect. The third display area (143) is larger than the display size of the markers (Mn, M1 to M4).

According to this aspect, by making the third display area (143) larger than the display size of the markers (Mn, M1 to M4), the image size of the captured image (A4) displayed in the third display area (143) can be increased.

In a fifth aspect of the information processing system (1), in any of the first to fourth aspects, a plurality of photographing positions (A2) are set on a spatial diagram (A1). The plurality of photographing positions (A2) include unphotographed positions where photographing has not yet been completed and photographed positions where photographing has been completed. The display unit (14) displays markers (Mn, M1 to M4) at the unphotographed positions on the spatial diagram (A1) by overlaying them at first display positions corresponding to the unphotographed positions, and displays object information (A31) related to the subject to be photographed at the unphotographed positions by overlaying it at a second display position corresponding to the first display position corresponding to the unphotographed positions. The display unit (14) displays markers (Mn, M1 to M4) at the photographed positions on the spatial diagram (A1) by overlaying them at first display positions corresponding to the photographed positions, and displays an image (A4) photographed at the photographed position by overlaying it at a second display position corresponding to the first display position corresponding to the photographed position.

According to this aspect, at unphotographed positions on the spatial map (A1), markers (Mn, M1-M4) are displayed superimposed at the first display position, and target information (A31) is displayed superimposed at the second display position, allowing the user to easily understand where and what target to photograph. Furthermore, at photographed positions on the spatial map (A1), markers (Mn, M1-M4) are displayed superimposed at the first display position, and the photographed image (A4) is displayed superimposed at the second display position, allowing the user to check the photographed image (A4) taken at the photographed position.

In the information processing system (1) of the sixth aspect, in the fifth aspect, the memory unit (12) further stores sequence information (A32) indicating the sequence in which photographs are taken at the plurality of photographing positions (A2). The display unit (14) displays the sequence information (A32) in association with markers (Mn, M1 to M4) at each of the plurality of first display positions.

According to this aspect, the user can easily understand the order in which to take photos at multiple shooting positions (A2).

The information processing system (1) of the seventh aspect, in any one of the first to sixth aspects, further includes an operation unit (13) that accepts a user's operation to switch the display mode of the display unit (14) between a simple display mode and a detailed display mode. When the display mode of the display unit (14) is the detailed display mode, the display unit (14) displays, in an unphotographed state, a spatial diagram (A1) with markers (Mn, M1 to M4) superimposed at the first display position and target information (A31) superimposed at the second display position. When the display mode of the display unit (14) is the simple display mode, the display unit (14) displays a spatial diagram (A1) with markers (Mn, M1 to M4) superimposed at the first display position.

According to this aspect, the user can switch the display mode of the display unit (14) between a simple display mode and a detailed display mode. In the detailed display mode, the user can easily understand where and what kind of subject to photograph based on the subject information (A31) displayed in the second display position.

In the information processing system (1) of the eighth aspect, in the seventh aspect, when the display mode of the display unit (14) is the detailed display mode, the display unit (14) displays a spatial diagram (A1) in which markers (Mn, M1 to M4) are superimposed at the first display position and the photographed image (A4) is superimposed at the second display position in a photographed state in which a photographed image (A4) taken at the photographing position (A2) exists.

According to this aspect, the user can switch the display mode of the display unit (14) between a simple display mode and a detailed display mode. In the detailed display mode, the user can check the captured image (A4) captured at the capturing position (A2) based on the captured image (A4) displayed at the second display position.

A ninth aspect of the information processing method is an information processing method for an information processing system (1) including a storage unit (12) and a display unit (14), and includes a display step. The storage unit (12) stores a spatial diagram (A1) that is a diagram of a target space, a shooting position (A2) set on the spatial diagram (A1), and target information (A31) related to the target to be photographed at the shooting position (A2). The display unit (14) displays the spatial diagram (A1), markers (Mn, M1-M4) corresponding to the shooting position (A2), and the target information (A31). In the display step, in an unphotographed state where photography at the shooting position (A2) is not yet completed, the spatial diagram (A1) is displayed with the markers (Mn, M1-M4) superimposed at a first display position corresponding to the shooting position (A2) and the target information (A31) superimposed at a second display position corresponding to the first display position.

This allows the user to easily understand where and what subject they should photograph.

In a tenth aspect of the information processing method, in the ninth aspect, a plurality of photographing positions (A2) are set on a spatial diagram (A1), and the plurality of photographing positions (A2) include unphotographed positions where photographing has not yet been completed and photographed positions where photographing has been completed. In the display step, markers (Mn, M1 to M4) are superimposed at the unphotographed positions on the spatial diagram (A1) at first display positions corresponding to the unphotographed positions, and object information (A31) related to the subject to be photographed at the unphotographed positions is superimposed and displayed at a second display position corresponding to the first display position corresponding to the unphotographed positions. In the display step, markers (Mn, M1 to M4) are superimposed at the photographed positions on the spatial diagram (A1) at first display positions corresponding to the photographed positions, and an image (A4) photographed at the photographed position is superimposed and displayed at a second display position corresponding to the first display position corresponding to the photographed position.

According to this aspect, at unphotographed positions on the spatial map (A1), markers (Mn, M1-M4) are displayed superimposed at the first display position, and target information (A31) is displayed superimposed at the second display position, allowing the user to easily understand where and what target to photograph. Furthermore, at photographed positions on the spatial map (A1), markers (Mn, M1-M4) are displayed superimposed at the first display position, and the photographed image (A4) is displayed superimposed at the second display position, allowing the user to check the photographed image (A4) taken at the photographed position.

The program of the eleventh aspect is a program for causing one or more processors of a computer system to execute the information processing method of the ninth or tenth aspect.

This allows the user to easily understand where and what subject they should photograph.

Not limited to the above aspects, various configurations (including modified examples) of the information processing system (1) according to the embodiment can be embodied as an information processing method performed by the information processing system (1), a (computer) program, or a non-transitory recording medium on which a program is recorded, etc.

The configurations according to the second to eighth aspects are not essential for the information processing system (1) and may be omitted as appropriate. Furthermore, the configuration according to the tenth aspect is not essential for the information processing method and may be omitted as appropriate.

1 Information processing system 12 Storage unit 13 Operation unit 14 Display unit 141 First display area 142 Second display area 143 Third display area A1 Space diagram A2 Shooting position A4 Shooting image A31 Object information A32 Order information Mn, M1 to M4 Markers

Claims (9)

a storage unit that stores a space map that is a diagram of a target space, a shooting position set on the space map, and target information related to a target to be photographed at the shooting position;
a display unit that displays the spatial map, markers corresponding to the photographing positions, and the target information ;
an operation unit that accepts a user's operation to switch the display mode of the display unit to either a simple display mode or a detailed display mode ,
The display unit
When the display mode of the display unit is the detailed display mode, in an unphotographed state where photography at the photography position has not yet been completed, the marker is superimposed on a first display position on the space map corresponding to the photography position, and the space map is displayed with the target information superimposed on a second display position corresponding to the first display position ;
When the display mode of the display unit is the simple display mode, the space diagram is displayed with the marker superimposed on the first display position.
Information processing system. a second display area for displaying the target information is smaller than a first display area for displaying the spatial diagram, and the second display area is larger than a display size of the marker;
The information processing system according to claim 1 . When the display mode of the display unit is the detailed display mode, in a photographed state in which a photographed image photographed at the photographing position exists, the display unit superimposes the marker at the first display position and displays the space diagram with the photographed image superimposed at the second display position.
The information processing system according to claim 1 . a third display area for displaying the captured image is smaller than a first display area for displaying the spatial diagram, and the third display area is larger than a display size of the marker;
The information processing system according to claim 3 . A plurality of the photographing positions are set on the spatial map,
The plurality of photographing positions include unphotographed positions where photographing has not been completed and photographed positions where photographing has been completed,
When the display mode of the display unit is the detailed display mode,
At the unphotographed position on the spatial diagram, the marker is superimposed at the first display position corresponding to the unphotographed position, and the target information regarding the photographed target to be photographed at the unphotographed position is superimposed at the second display position corresponding to the first display position corresponding to the unphotographed position, and
At the photographed position on the space map, the marker is superimposed at the first display position corresponding to the photographed position, and the photographed image photographed at the photographed position is superimposed and displayed at the second display position corresponding to the first display position corresponding to the photographed position.
The information processing system according to claim 1 . the storage unit further stores sequence information indicating a sequence of taking images at the plurality of imaging positions;
the display unit displays the order information in association with the marker at each of the plurality of first display positions.
The information processing system according to claim 5 . a storage unit that stores a space map that is a diagram of a target space, a shooting position set on the space map, and target information related to a target to be photographed at the shooting position;
an information processing method for an information processing system including a display unit that displays the spatial map, a marker corresponding to the shooting position, and the target information,
an operation receiving step of receiving a user's operation to switch the display mode of the display unit to either a simple display mode or a detailed display mode;
a display step,
In the display step,
When the display mode of the display unit is the detailed display mode, in an unphotographed state where photography at the photography position has not yet been completed, the marker is superimposed on a first display position on the space map corresponding to the photography position, and the space map is displayed with the target information superimposed on a second display position corresponding to the first display position;
When the display mode of the display unit is the simple display mode, the space diagram is displayed with the marker superimposed on the first display position.
Information processing methods. A plurality of the photographing positions are set on the spatial map,
The plurality of photographing positions include unphotographed positions where photographing has not been completed and photographed positions where photographing has been completed,
In the display step, when the display mode of the display unit is the detailed display mode,
At the unphotographed position on the spatial diagram, the marker is superimposed at the first display position corresponding to the unphotographed position, and the target information regarding the photographed target to be photographed at the unphotographed position is superimposed at the second display position corresponding to the first display position corresponding to the unphotographed position, and
At the photographed position on the space map, the marker is superimposed at the first display position corresponding to the photographed position, and the photographed image photographed at the photographed position is superimposed and displayed at the second display position corresponding to the first display position corresponding to the photographed position.
The information processing method according to claim 7. 9. The information processing method according to claim 7 or 8, wherein the information processing method is executed by one or more processors of a computer system.
Program.