JP7579568B2 - Information processing system, information processing program, information processing device, and information processing method - Google Patents

Description

The present invention relates to an information processing system, an information processing program, an information processing device, and an information processing method.

Technology has been used in the past to display additional information related to an object on an image of the object captured by an imaging means. For example, Patent Document 1 discloses a technology that displays the name of a train line that can be boarded from a station, superimposed on a captured image of the station entrance.

JP 2005-37181 A

However, conventional technology has been insufficient in improving the convenience of using trains. For example, the presentation of information using augmented reality to users of public transport has been insufficient.

The present invention provides an information processing system, an information processing program, an information processing device, and an information processing method that can improve the convenience of using public transportation.

The information processing system according to the present invention comprises:
An identification means for identifying a location of an entrance/exit that can connect to a public transportation facility based on an image captured by an imaging means of the entrance/exit;
A generating means for generating guidance information for the public transportation facility taking into consideration the location of the specified entrance/exit;
an output means for outputting the generated guidance information;
The present invention is characterized by comprising:

The present invention can improve the convenience of using public transportation.

1 is a diagram showing a schematic configuration of an information processing system according to a first embodiment; 5 is a flowchart illustrating an example of an operation of the information processing system according to the first embodiment. FIG. 11 is a diagram showing an entrance/exit image obtained by capturing an image of a station entrance, as an example of the operation of the information processing system according to the first embodiment. FIG. 13 is a diagram showing guidance information superimposed and displayed on an entrance/exit image as an example of the operation of the information processing system according to the first embodiment. FIG. 11 is a diagram showing a dedicated guide screen for a station entrance/exit as an example of the operation of the information processing system according to the first embodiment. 10 is a flowchart illustrating an example of an operation of an information processing system according to a second embodiment. FIG. 11 is a diagram showing guidance information superimposed on a departure sign image obtained by capturing an image of a departure sign at a station, as an example of the operation of the information processing system according to the second embodiment. FIG. 13 is a diagram showing guidance information and related information superimposed on a departure sign image obtained by capturing an image of a departure sign at a station, as a modified example of the operation of the information processing system according to the second embodiment. FIG. 11 is a diagram showing guidance information displayed in a foreign language as a modified example of the operation of the information processing system according to the second embodiment.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that in each drawing, components having equivalent functions are given the same reference numerals, and detailed descriptions of components with the same reference numerals will not be repeated.

(First embodiment)
An information processing system 1 according to a first embodiment of the present invention will be described. The information processing system 1 according to the first embodiment is a system that provides public transportation guide information to users (hereinafter, simply referred to as "users") of electronic devices such as mobile terminals (smartphones, mobile phones, tablet terminals, etc.) and wearable terminals (glasses-type, wristwatch-type, earphone-type, etc.).

As shown in FIG. 1, the information processing system 1 includes a terminal device 2, which corresponds to the electronic device described above, and a server 3. The terminal device 2 and the server 3 are connected to each other so as to be able to communicate with each other via a network such as the Internet.

The network may be either a wired line or a wireless line, and the type and form of the line are not important. At least a portion of the terminal device 2 and the server 3 are realized by a computer.

The terminal device 2 is used by the user to receive guidance information for public transportation, and is, for example, an information processing terminal such as a mobile terminal or a wearable terminal. The terminal device 2 may be composed of a single device, or may be composed of multiple devices. When the terminal device 2 is composed of multiple devices, the terminal device 2 may be composed of, for example, a mobile terminal (e.g., a smartphone, etc.) that can be communicatively connected to the server 3, and a wearable terminal that can be communicatively connected to the mobile terminal, or may be composed of a mobile terminal and a camera that can be connected to the mobile terminal.

As shown in FIG. 1, the terminal device 2 has a communication unit 21, an input unit 23, an output unit 24, an imaging unit 25 which is an example of an imaging means, and a control unit 22.

The communication unit 21 is an interface for transmitting and receiving information between the control unit 22 and the server 3 via the network.

The input unit 23 is an interface for the user to input information to the terminal device 2, and may be, for example, a touch panel, microphone, touch pad, dial buttons, or gestures on a mobile terminal.

The output unit 24 is an interface that outputs various information from the terminal device 2 to the user, and is, for example, a video display means such as a liquid crystal display. As an example, the output unit 24 displays a GUI (Graphical User Interface) for receiving operations from the user via the input unit 23. The output unit 24 may also have a speaker or earphones that output various information as audio.

Note that the output unit 24 does not have to directly present information to the user. For example, the output unit 24 may output a video signal or an audio signal to a video display means or an audio output means connected to the outside of the terminal device 2, may output data to an externally connected printing device, or may output data to a storage device within the terminal device 2 or an external storage device for storage.

The imaging unit 25 is a device that inputs an image of an object to the terminal device 2, and is, for example, a camera equipped with a solid-state image sensor such as a CCD. The imaging unit 25 may be built into the main body of the terminal device 2, or may be connected to the main body of the terminal device 2 via an interface. In the first embodiment, the imaging unit 25 is used to capture an image of an entrance/exit that can connect to a public transportation facility.

As shown in FIG. 1, the control unit 22 has a guidance information presentation unit 221, which is an example of an output means. The guidance information presentation unit 221 presents the guidance information for public transportation transmitted from the server 3 to the user by outputting the information via the output unit 24.

The control unit 22 also controls the transmission of the captured image input from the imaging unit 25 to the server 3 via the communication unit 21. The control unit 22 may be realized by a processor in the terminal device 2 executing a predetermined program.

Next, the server 3 will be described. As shown in FIG. 1, the server 3 has a communication unit 31, a control unit 32, and a storage unit 33. Before describing the control unit 32, the communication unit 31 and the storage unit 33 will be described.

The communication unit 31 is an interface that transmits and receives information between the terminal device 2 and the control unit 32 of the server 3 via the network.

The storage unit 33 is, for example, a data storage such as a hard disk, and stores various databases. Note that the storage unit 33 does not necessarily have to be provided within the server 3, but may be provided within another device that is communicatively connected to the server 3 via a network.

The storage unit 33 has a route network information database 331. The route network information database 331 is a database for route search, and includes, for example, transportation network information as route network information. The transportation network information is information that specifies transportation networks such as railroads and buses, and road networks. The transportation network information includes public transportation route information, timetable information, fare information, and the like. Furthermore, the transportation network information is expressed by a combination of data on nodes on a route network representation, such as stations, and data on links, which are rail sections between the nodes. Furthermore, the road network information is expressed by a combination of data on nodes on a road network representation, such as intersections, and data on links, which are road sections between the nodes.

Although not shown, the memory unit 33 may also have a map information database used for displaying maps and searching for locations. The map information may include map data such as national or regional road maps, and may also include map object information (note information, symbol information, etc.) associated with the map data. Furthermore, the memory unit 33 may also have a POI (Point of Interest) information database used for searching for facilities and setting destinations. POI information is information on facilities and events. The POI information includes name information and location information on facilities and events. The POI information may also include information on entrances and exits that can connect to public transportation facilities such as stations. The memory unit 33 may also have station layout information that shows station layouts.

Next, the control unit 32 of the server 3 will be described. As shown in FIG. 1, the control unit 32 has a specification unit 321, which is an example of a specification means, and a guidance information generation unit 322, which is an example of a generation means.

The identification unit 321 identifies the location of the entrance/exit on the entrance/exit image based on an image captured by the imaging unit 25 of an entrance/exit that can connect to a public transportation facility (hereinafter also referred to as an entrance/exit image).

Here, public transportation facilities include, for example, train stations and bus terminals. Also, entrances that can connect to public transportation facilities are not limited to entrances of the public transportation facilities themselves, but also include entrances of facilities such as commercial buildings that are directly connected to stations.

The identification unit 321 may determine whether or not a facility on the entrance/exit image is an entrance/exit based on an algorithm constructed by learning images of entrances/exits that can connect to public transportation facilities throughout the country. If it is determined to be an entrance/exit, the identification unit 321 may determine which location the determined entrance/exit is, using a VPS (Visual Positioning Service), a GPS, or a combination of these. The location of the entrance/exit may be determined, for example, in the form of "A1 entrance/exit at Station A," "B1 entrance/exit at Station B," or the like. After determining the location of the entrance/exit using the VPS or GPS, the identification unit 321 may more accurately determine the location of the entrance/exit by extracting text information, etc., using image recognition, as necessary.

The guidance information generating unit 322 generates guidance information for public transportation facilities taking into consideration the locations of the entrances and exits identified by the identifying unit 321. The guidance information is not limited to guidance information regarding the use of the public transportation itself, such as timetables, but may also be guidance information regarding the use of facilities of the public transportation, such as barrier-free facilities (e.g., elevators, escalators, ramps, etc.), toilets, ATMs, and stores.

More specifically, the guidance information that takes into account the location of the identified entrance/exit is information that takes into account the identified entrance/exit as the starting point for travel (walking, travel by wheelchair, etc.) to use public transportation. Here, the use of public transportation is not limited to the use of the public transportation itself, but may also be the use of the facilities of the public transportation.

As an example, the guidance information may be information that takes into account the time required from a specified entrance/exit to the boarding/alighting location or ticket gate of the public transportation. More specifically, the guidance information may be a timetable of public transportation that can be boarded within the required time. The timetable may be a timetable for each line. If there are multiple lines, the lines for which the timetable is displayed may be narrowed down based on user information (information such as lines, routes, and commuter passes registered in advance by the user), the order in which the timetable is displayed may be changed, or it may be highlighted. By outputting a timetable of public transportation that can be boarded within the required time as guidance information, the user can easily understand that they will be able to catch the train shown in the timetable.

As another example, the guidance information may be a map of the facility of the public transport system showing the specified entrance/exit in an identifiable manner. In this case, the specified entrance/exit may be placed at the center of the map or highlighted so as to be easily identifiable. The guidance information regarding the use of the public transport system facilities described above (information on barrier-free facilities, etc.) may be displayed as an icon corresponding to the corresponding position on the map. By outputting a map of the facility showing the specified entrance/exit in an identifiable manner as the guidance information, the user can easily understand whether there is an escalator or elevator at the specified entrance/exit, whether there is a toilet outside the ticket gate, whether an entrance/exit other than the specified entrance/exit is closer to the platform, etc.

As another example, the guidance information may be an indoor route from the specified entrance/exit to the platform, which is the boarding/disembarking location of the selected line. The selection of the route may be performed by the information processing system 1 (e.g., the guidance information generating unit 322) according to predetermined selection conditions. The route selected in generating the indoor route (i.e., the route selection conditions) may be, for example, a route or a route on the route registered in advance in the terminal device 2 by a user operation using the input unit 23, or a route indicated in electronic ticket information (e.g., a commuter pass, etc.) purchased in advance by the user and registered in the terminal device 2. By displaying the indoor route from the specified entrance/exit to the boarding/disembarking location of the selected line as the guidance information, the user can easily understand which route to take to get to the platform.

The guidance information presentation unit 221 may superimpose the guidance information generated by the guidance information generation unit 322 on the captured image via the output unit 24. The guidance information presentation unit 221 may also transition from the entrance/exit image to a dedicated entrance/exit guidance image including the guidance information. Alternatively, the guidance information presentation unit 221 may switch between displaying the entrance/exit image with the guidance information superimposed thereon and the dedicated entrance/exit guidance image.

In addition, the guidance information may be displayed in a display mode that can be selected by a user operation using the input unit 23. The selection of the guidance information is not limited to manual input, but may be performed by voice input or the like. When the guidance information is selected, the guidance information generation unit 322 may generate related information related to the guidance information, and the guidance information presentation unit 221 may display the generated related information on the output unit 24. For example, in a state in which a premises map of a public transportation facility is displayed as guidance information, when an icon indicating a barrier-free facility displayed in a selectable manner on the premises map is selected by a user operation, the guidance information generation unit 322 may generate, as related information, a guidance route from the specified entrance/exit to the barrier-free facility indicated by the selected icon or the nearest entrance/exit to the barrier-free facility. In this case, the guidance information presentation unit 221 may display the guidance route on the premises map, or may display it superimposed on the actual image. According to such a configuration, for example, in the case of movement by a wheelchair or a stroller, it is possible to easily understand that there is an elevator at another entrance, although there is no elevator at the entrance that is being imaged, and convenience for the user can be improved by displaying a guide route from the imaged point to an entrance with an elevator. In addition, for example, in a state in which an indoor route from the specified entrance to the platform, which is the boarding and alighting place of the selected line, is displayed in a selectable display mode as guidance information, when the displayed indoor route is selected by a user operation, the guidance information generating unit 322 may display detailed information of the indoor route or options of other indoor routes as related information. Examples of detailed information of the indoor route include the distance of the indoor route, the ventilation properties such as the width and ceiling height of the indoor route, and the degree of congestion of the indoor route based on the GPS information of the other terminal. According to such a configuration, it is possible to know in advance whether the indoor route can be passed smoothly and safely, and it is also possible to change to another indoor route as necessary.

In addition, the output of guidance information is not limited to image display only, but may also include audio output.

The control unit 32 executes control to transmit the guidance information generated by the guidance information generation unit 322 to the terminal device 2 via the communication unit 31. The control unit 32 may be realized by a processor in the server 3 executing a predetermined program.

(Example of operation)
Next, a description will be given of an example of the operation of the information processing system 1 according to the first embodiment. Fig. 2 is a flowchart showing an example of the operation of the information processing system 1 according to the first embodiment.

As shown in FIG. 2, first, the imaging unit 25 of the terminal device 2 captures an entrance/exit image by capturing an entrance/exit that can connect the user to a public transportation facility (step S21). FIG. 3 is a diagram showing an entrance/exit image i1 obtained by capturing an entrance/exit of a station, which is an example of a public transportation facility, with the imaging unit 25 as an example of the operation of the information processing system 1 according to the first embodiment. When a station name ("Station A" in FIG. 3) is reflected in the image as in the entrance/exit image i1 shown in FIG. 3, for example, it is possible to determine that the facility in the image is an entrance/exit based on the learning results of machine learning of the character information of the station name, and then perform image analysis of the characters and marks in the image to identify which entrance/exit of which public transportation facility the entrance/exit is. Even if the station name is not reflected in the captured image, the location of the entrance/exit can be uniquely identified by the VPS, GPS, or a combination thereof.

After the entrance/exit image is acquired, the control unit 22 of the terminal device 2 transmits the acquired entrance/exit image to the server 3 via the communication unit 21 (step S22).

After the entrance/exit image is transmitted, the identification unit 321 of the server 3 identifies the location of the entrance/exit on the entrance/exit image based on the transmitted entrance/exit image (step S31). As described above, the location of the entrance/exit can be identified based on the results of learning the entrance/exit image, or by estimating the current position by combining the VPS and GPS.

After the location of the entrance/exit is specified, the guidance information generating unit 322 of the server 3 generates guidance information for the public transportation facility taking into consideration the location of the specified entrance/exit (step S32). Specifically, the guidance information generating unit 322 generates information as guidance information taking into consideration the specified entrance/exit as a starting point for starting movement to use the public transportation facility. As an example, the guidance information generating unit 322 generates information taking into consideration the required time from the specified entrance/exit to the boarding/alighting location or ticket gate of the public transportation facility. More specifically, the guidance information generating unit 322 generates a timetable of the public transportation facility that allows boarding within the required time from the specified entrance/exit to the boarding/alighting location or ticket gate of the public transportation facility. More specifically, the guidance information generating unit 322 generates a timetable of the public transportation facility for each route that allows boarding within the required time from the specified entrance/exit to the boarding/alighting location or ticket gate of the public transportation facility. As another example, the guidance information generating unit 322 generates a premises map of the public transportation facility that identifiably shows the specified entrance/exit. As another example, the guidance information generating unit 322 generates an indoor route from the identified entrance/exit to the boarding/alighting location of the selected line.

After the guidance information is generated, the control unit 32 of the server 3 transmits the generated guidance information to the terminal device 2 via the communication unit 31 (step S33).

After the guidance information is transmitted, the guidance information presentation unit 221 of the terminal device 2 outputs the transmitted guidance information via the output unit 24 (step S23). When displaying the guidance information on the screen of the terminal device 2, the guidance information presentation unit 221 may appropriately adjust the character size and display amount of the guidance information according to the screen size.

Figure 4 is a diagram showing guidance information i2 superimposed on an entrance/exit image i1 as an example of the operation of the information processing system 1 according to the first embodiment. For example, as shown in Figure 4, the guidance information presentation unit 221 outputs guidance information by superimposing the guidance information i2 on the entrance/exit image i1. In the example shown in Figure 4, the guidance information i2 includes timetable information i21 showing train timetables for each line, and station layout information i22 showing a station layout map.

The time information i21 is image information generated to indicate the departure time of the nearest train that can be boarded within the required time, taking into consideration the time required from the entrance/exit location on the entrance/exit image i1 to the station platform. Therefore, even if the nearest departure time is at the current time or the time when the entrance/exit is imaged, the time information i21 does not include the time that departs before the required time to the platform has elapsed, or the user must scroll to display the departure time. The time information i21 also includes the required time from the entrance/exit to the platform (e.g., 2 minutes for line Z) calculated based on the route network information. By displaying such time information i21, the user can easily understand how long it will take to reach the station platform and whether they will make it to the departure time in the displayed timetable. In the example shown in FIG. 4, the time information i21 also includes train delay information (e.g., 5 minutes delay), so that the user can easily understand that the departure time will be delayed.

The station interior map information i22 is image information of a station interior map generated to show the entrance/exit on the entrance/exit image i1 in a identifiable state, taking into consideration that the location of the entrance/exit on the entrance/exit image i1 is used as a starting point for starting movement to use a train. As shown in FIG. 4, in the station interior map information i22, the location P of the entrance/exit on the entrance/exit image i1 is conspicuously displayed at the center of the station interior map i221 by a circular mark or the like. In addition, the station interior map information i22 includes multiple types of icons i222 indicating barrier-free facilities. The icons i222 can be selected by a user operation, and the guidance information generating unit 322 generates guidance information i2 so that the selected icon i223 is displayed in association with the corresponding position on the station interior map i221. Furthermore, the icon i223 displayed in association with the station interior map i221 can be selected by a user operation, and when an icon i223 is selected, the guidance information generating unit 322 generates a guidance route from the specified entrance/exit P to the barrier-free facility indicated by the selected icon i223. By displaying such station interior map information i22, the user can easily understand whether barrier-free facilities are available at the entrance/exit on the entrance/exit image i1, and how to get to the entrance/exit where the barrier-free facilities are available. Note that the station interior map information i22 may also include icons indicating station facilities other than barrier-free facilities.

5 is a diagram showing a dedicated guidance screen S for a station entrance/exit as an example of the operation of the information processing system 1 according to the first embodiment. For example, as shown in FIG. 5, the guidance information presenting unit 221 may switch the display from the display screen of the guidance information i2 in FIG. 4 to the dedicated guidance screen S in FIG. 5 in accordance with a user operation, or may automatically switch without a user operation. The dedicated guidance screen S is image information generated by the guidance information generating unit 322 so as to include information similar to the guidance information i2. The dedicated guidance screen S may be a guidance screen on a dedicated app provided on a mobile terminal, or a guidance screen on the web. In addition to information similar to the guidance information i2, the dedicated guidance screen S may include unique information. For example, the dedicated guidance screen S may include an operation button (the "search" button in FIG. 5) that instructs a route search from the station corresponding to the identified entrance/exit to the destination. The dedicated guidance screen S is convenient because it can be continuously displayed without continuously capturing the station entrance/exit with the imaging unit 25.

As described above, according to the first embodiment, the location of an entrance/exit is identified based on an image capturing an entrance/exit that can connect to a public transportation facility, and public transportation guidance information that takes into account the identified entrance/exit location is generated and output, making it possible to provide sufficient information using augmented reality to users of public transportation. This can improve convenience when using public transportation.

Second Embodiment
Next, a second embodiment of the present invention will be described, which provides guidance information accompanying a captured image of a departure sign of a public transportation facility. The configuration of an information processing system 1 according to the second embodiment will be described below with reference to FIG.

As in the first embodiment, the information processing system 1 according to the second embodiment includes a terminal device 2 and a server 3 connected to each other via a network so as to be able to communicate with each other. In the second embodiment, the terminal device 2 also includes a communication unit 21, a control unit 22 having a guidance information presentation unit 221, an input unit 23, an output unit 24, and an imaging unit 25, and the server 3 includes a communication unit 31, a control unit 32 having an identification unit 321 and a guidance information generation unit 322, and a storage unit 33 having a route network information database 331. However, in the second embodiment, the content of the guidance information is different from that in the first embodiment, and therefore the functions of the components of the terminal device 2 and the server 3 are different from those in the first embodiment.

Specifically, the identification unit 321 of the server 3 determines whether the imaged object is a departure sign based on an image (hereinafter also referred to as a departure sign image) of a departure sign of a public transportation facility captured by the imaging unit 25, and identifies the public transportation facility where the departure sign is installed. As in the first embodiment, the public transportation facility may be a station or a bus stop.

The identification unit 321 may determine whether the captured object is a departure sign, for example, based on an algorithm constructed by learning images of departure signs at public transportation facilities throughout the country. If it is determined to be a departure sign, the identification unit 321 may identify the public transportation facility where the departure sign on the departure sign image is installed. The identification unit 321 may identify the public transportation facility where the departure sign on the departure sign image is installed by using a VPS, a GPS, or a combination of these. For example, the identification unit 321 may identify the public transportation facility where the departure sign is installed based on the platform number, route name, departure time, etc. indicated on the departure sign.

The guidance information generation unit 322 generates guidance information that includes at least information on departure times after the departure time shown on the departure sign image at the public transportation facility identified by the identification unit 321. The guidance information presentation unit 221 of the terminal device 2 outputs the guidance information via the output unit 24. The guidance information presentation unit 221 may superimpose and display the guidance information on the departure sign image in a display mode that is continuous with the end of the departure sign on the departure sign image. By displaying such guidance information, the user can easily grasp the subsequent departure information of the departure sign.

The guidance information may further include at least one of operation information and congestion information for the public transportation facility corresponding to the departure time indicated on the departure board. The guidance information may also further include at least one of operation information and congestion information for the public transportation facility corresponding to a departure time after the departure time indicated on the departure board. By displaying such guidance information, the user can easily determine whether or not to board the public transportation facility at the departure time indicated on the departure board or guidance information.

The guidance information may not include information about departure times after the departure time shown on the departure board, but may include only information related to the departure board other than the departure time. Information related to the departure board other than the departure time may be, for example, train congestion (such as congestion level per train or congestion per car) and operation information (such as delays) and location shown on the departure board, and this information may be superimposed (displayed near the departure board) in imitation of the departure board.

The guidance information presentation unit 221 may switch between displaying a departure sign image with guidance information superimposed thereon and a dedicated guidance screen for the departure sign. The dedicated guidance screen is, for example, a screen including the departure time shown on the departure sign and the departure time shown in the guidance information, and can be continuously displayed without continuously capturing the departure sign with the imaging unit 25. The guidance information presentation unit 221 may also display the guidance information in a selectable display mode, and when guidance information is selected, the guidance information generation unit 322 may generate related information related to the selected guidance information and display it on the guidance information presentation unit 221. The related information may be, for example, information indicating the current location of the public transportation facility at the departure time shown in the selected guidance information.

(Example of operation)
Next, a description will be given of an example of the operation of the information processing system 1 according to the second embodiment. Fig. 6 is a flowchart showing an example of the operation of the information processing system 1 according to the second embodiment.

As shown in FIG. 6, first, the imaging unit 25 of the terminal device 2 captures an image of the departure sign of the public transportation system to obtain the departure sign image (step S211).

After the departure mark image is acquired, the control unit 22 of the terminal device 2 transmits the acquired departure mark image to the server 3 via the communication unit 21 (step S212).

After the departure sign image is transmitted, the identification unit 321 of the server 3 identifies the public transport facility where the departure sign on the departure sign image is located based on the transmitted departure sign image (step S311). As described above, the public transport facility can be identified based on the results of learning the departure sign image and current position estimation using a combination of VPS and GPS.

After the public transportation facility is identified, the guidance information generation unit 322 of the server 3 generates guidance information that provides information about departure times after the departure time shown on the departure sign image at the identified facility (step S312).

After the guidance information is generated, the control unit 32 of the server 3 transmits the generated guidance information to the terminal device 2 via the communication unit 31 (step S313).

After the guidance information is transmitted, the guidance information presentation unit 221 of the terminal device 2 outputs the transmitted guidance information via the output unit 24 (step S213).

Figure 7 is a diagram showing guidance information i4 superimposed on a departure sign image i3 as an example of the operation of the information processing system 1 according to the second embodiment. For example, as shown in Figure 7, the guidance information presentation unit 221 outputs guidance information by superimposing the guidance information i4 on a departure sign image i3 that is an image of a departure sign at a station. In the example shown in Figure 7, the guidance information i4 includes departure time information i41 indicating a departure time after the latest departure time (10:25 in Figure 7) among the departure times indicated in the departure sign image i3, and congestion information i42 indicating the degree of congestion of the train corresponding to the departure time indicated in the departure sign image i3 and the degree of congestion of the train corresponding to the departure time indicated in the departure time information i41.

By displaying the departure time information i41, the user can easily grasp the subsequent departure time of the departure sign. By displaying the congestion information i42, the user can easily grasp the degree of congestion of the train corresponding to the departure sign and the train corresponding to the departure time information i41.

Figure 8 is a diagram showing related information i5 as an example of the operation of the information processing system 1 according to the second embodiment. As shown in Figure 8, when the departure time (10:34 in Figure 8) shown in the departure time information i41 is selected by a user operation, the guidance information generation unit 322 generates related information i5 indicating the current running position of the train at the selected departure time, and the guidance information presentation unit 221 may superimpose the generated related information i5 on the departure sign image i3.

As shown in FIG. 9, the guidance information i4 may be displayed in a foreign language, not limited to Japanese. In this case, as shown in FIG. 9, the departure time may be superimposed in a foreign language on the departure time on the departure sign image. The guidance information generating unit 322 may determine whether or not to display the guidance information in a foreign language based on, for example, language information set in the OS of the terminal that captures the image, language information on an app, or the recognition result of a language input by voice.

According to the second embodiment, a public transportation facility where a departure sign is installed is identified based on an image of the public transportation facility's departure sign, and guidance information is generated and output that provides information about departure times after the departure time indicated on the departure sign at the identified public transportation facility, thereby providing sufficient information using augmented reality to users of public transportation. This can improve the convenience of using public transportation.

In the above-described embodiment, since there may be cases where there is no departure sign on the station platform or the departure sign is too far away to be recognized, the guidance information may be output using a similar means when a direction sign or station sign other than a departure sign is captured. Specifically, the identification unit 321 may identify the facility of the public transportation facility where the direction sign or station sign is installed by performing image recognition of the direction indicated by the direction sign or the station name indicated by the station sign based on the captured image of the direction sign or station sign of the public transportation facility captured by the imaging unit 25. The guidance information generation unit 322 may generate information on the departure time after the current time at the facility of the public transportation facility identified based on the captured image of the direction sign or station sign as guidance information, or may generate guidance information that includes at least information on the departure time after the departure time indicated by the departure sign. The guidance information generation unit 322 may also generate guidance information that includes information other than the departure time (for example, the degree of congestion of trains arriving at the identified station, delay information, location information, etc.). Furthermore, the generated guidance information may be displayed near directional signs or station signs in the captured image.

At least a part of the information processing system described in the above embodiment may be configured with hardware or software. The hardware constituting the information processing system may include a sensor such as LiDAR (light detection and ranging), which is an example of a sensor using laser light. The sensor may be installed (built-in) as standard in the terminal device 2, or may be connected to the terminal device 2 as an add-on. The sensor may detect, for example, the shape of an object, the position of the object, and the distance to the object. The detection information of the sensor may be used to assist in determining whether the object captured by the imaging unit 25 is an entrance or exit, or in determining which of multiple entrances that can connect to a public transportation facility the captured object is. The detection information of the sensor may also be used to assist in determining at what position, in what range, or with what level of detail the guidance information is to be displayed in the captured image. The detection information of the sensor may also be used to assist in determining whether the captured image and the dedicated guidance image are automatically switched. When the information processing system is configured with software, a program that realizes at least some of the functions of the information processing system may be stored on a recording medium such as a flexible disk or CD-ROM, and may be read and executed by a computer. The recording medium is not limited to removable recording media such as magnetic disks and optical disks, but may also be fixed recording media such as a hard disk device or memory.

In addition, a program that realizes at least a part of the functions of the information processing system may be distributed via a communication line (including wireless communication) such as the Internet. Furthermore, the program may be encrypted, modulated, or compressed and distributed via a wired line or wireless line such as the Internet, or stored on a recording medium.

Furthermore, the information processing system may be operated by one or more information processing devices. When multiple information processing devices are used, one of the information processing devices may be a computer, and the computer may execute a predetermined program to realize the function as at least one of the means of the information processing system.

In addition, in the method invention, all processes (steps) may be performed automatically by a computer. Alternatively, each process may be performed by a computer, while progress control between processes may be performed manually. Furthermore, at least some of the processes may be performed manually.

Based on the above description, a person skilled in the art may be able to conceive of additional effects and various modifications of the present invention, but the aspects of the present invention are not limited to the individual embodiments described above. Various additions, modifications, and partial deletions are possible within the scope that does not deviate from the conceptual idea and intent of the present invention derived from the contents defined in the claims and their equivalents.

1 Information processing system 2 Terminal device 221 Guide information presentation unit 3 Server 321 Identification unit 322 Guide information generation unit

Claims (26)

An identification means for identifying a location of an entrance/exit that can connect to a public transportation facility based on an image captured by an imaging means of the entrance/exit;
A generating means for generating guidance information for the public transportation facility taking into consideration the location of the specified entrance/exit;
an output means for displaying the generated guidance information on the captured image by superimposing the information on the captured image or on a dedicated guidance screen ;
Equipped with
The information processing system is characterized in that the output means switchably displays the captured image on which the guidance information is superimposed and the dedicated guidance screen . The information processing system according to claim 1, characterized in that the guidance information is information that considers the identified entrance/exit as a starting point for travel to use the public transportation system. The information processing system according to claim 2, characterized in that the guidance information takes into account the time required to travel from the specified entrance/exit to the boarding/alighting location or ticket gate of the public transportation facility. The information processing system according to claim 3, characterized in that the guidance information is a timetable for public transportation for each route that can be boarded within the required time. The information processing system according to claim 4, characterized in that if the entrance is an entrance of a facility directly connected to the public transportation facility, the guidance information includes a mark indicating at least one of the public transportation facility and the route of the public transportation facility. The information processing system according to claim 2, characterized in that the guidance information is a map of the facility of the public transportation facility that identifiably shows the specified entrance/exit. The information processing system according to claim 2, characterized in that the guidance information is an indoor route from the specified entrance/exit to the boarding/alighting location of the selected line. The information processing system according to any one of claims 2 to 7, characterized in that the output means displays the guidance information in a selectable display mode, and when the guidance information is selected, displays related information related to the guidance information. an identification means for identifying a facility of the public transportation facility in which a departure board is installed based on an image captured by an imaging means of the departure board of the public transportation facility;
a generation means for generating guidance information including at least information on departure times after the departure time indicated on the departure board at the specified public transportation facility;
an output means for outputting the generated guidance information;
An information processing system comprising: The information processing system according to claim 9 , wherein the output means displays the guidance information superimposed on the captured image in a display manner continuous with the departure board. 11. The information processing system according to claim 10 , wherein the output means switchably displays the captured image with the guidance information superimposed thereon and a dedicated guidance screen for the departure board. An information processing system according to any one of claims 9 to 11, characterized in that the guidance information further includes operation information and/or congestion information of the public transportation system corresponding to at least one of the departure time indicated on the departure board and departure times after the departure time. The information processing system according to any one of claims 9 to 12 , characterized in that the output means displays the guidance information in a selectable display form, and when the guidance information is selected, displays related information related to the guidance information. the generating means generates second guidance information for providing guidance on a guidance target other than a departure time indicated on the departure board at the identified public transportation facility;
10. The information processing system according to claim 9 , wherein the output means outputs the generated second guidance information. Computer,
An identification means for identifying a location of an entrance/exit that can connect to a facility of public transportation based on an image captured by an imaging means of the entrance/exit;
a generating means for generating guidance information for the public transportation facility taking into consideration the location of the specified entrance/exit; and an output means for displaying the generated guidance information by superimposing it on the captured image or on a dedicated guidance screen .
Function as a
The information processing program is characterized in that the output means displays the captured image on which the guidance information is superimposed and the dedicated guidance screen in a switchable manner . An information processing system including a plurality of computers connected in a communicative manner,
An identification means for identifying a location of an entrance/exit that can connect to a public transportation facility based on an image captured by an imaging means of the entrance/exit;
A generating means for generating guidance information for the public transportation facility taking into consideration the location of the specified entrance/exit;
an output means for displaying the generated guidance information on the captured image by superimposing the information on the captured image or on a dedicated guidance screen ;
Equipped with
The output means functions as an information processing system that switchably displays the captured image on which the guidance information is superimposed and the dedicated guidance screen ,
An information processing program for causing at least one of the computers to function as at least one of the means. In order to make the information processing system according to any one of claims 1 to 8 function using a plurality of computers connected in a communicable manner,
9. An information processing program for causing one of said computers to function as at least one of the means in the information processing system according to any one of claims 1 to 8 . 9. An information processing program for causing a computer to function as at least one of the means in the information processing system according to claim 1. By a plurality of information processing devices connected to each other so as to be able to communicate with each other,
An identification means for identifying a location of an entrance/exit that can connect to a public transportation facility based on an image captured by an imaging means of the entrance/exit;
A generating means for generating guidance information for the public transportation facility taking into consideration the location of the specified entrance/exit;
an output means for displaying the generated guidance information on the captured image by superimposing the information on the captured image or on a dedicated guidance screen ;
Equipped with
In order to configure an information processing system in which the captured image and the dedicated guidance screen are switchably displayed with the guidance information superimposed thereon, the output means
An information processing device comprising at least one of the above means. A step in which a specifying means specifies a location of an entrance/exit that can connect to a facility of public transportation based on an image captured by an imaging means of the entrance/exit;
A step of generating guidance information for the public transportation facility by a generating means, the guidance information taking into consideration the location of the identified entrance/exit;
an output means for displaying the generated guidance information by superimposing the generated guidance information on the captured image or on a dedicated guidance screen ;
Equipped with
The information processing method, wherein the output means switchably displays the captured image on which the guidance information is superimposed and the dedicated guidance screen . Computer,
an identification means for identifying a facility of the public transportation facility in which a departure board is installed based on an image captured by an imaging means of the departure board of the public transportation facility;
a generating means for generating guidance information including at least information on departure times after the departure time indicated on the departure board at the specified public transportation facility; and an output means for outputting the generated guidance information.
An information processing program characterized by causing the program to function as follows. An information processing system including a plurality of computers connected in a communicative manner,
an identification means for identifying a facility of the public transportation facility in which a departure board is installed based on an image captured by an imaging means of the departure board of the public transportation facility;
a generation means for generating guidance information including at least information on departure times after the departure time indicated on the departure board at the specified public transportation facility;
an output means for outputting the generated guidance information;
In order to make the information processing system function,
An information processing program for causing at least one of the computers to function as at least one of the means. In order to make the information processing system according to any one of claims 9 to 14 function using a plurality of computers connected in a communicable manner,
15. An information processing program for causing one of said computers to function as at least one of the means in the information processing system according to any one of claims 9 to 14 . 15. An information processing program for causing a computer to function as at least one of the means in the information processing system according to any one of claims 9 to 14 . By a plurality of information processing devices connected to each other so as to be able to communicate with each other,
an identification means for identifying a facility of the public transportation facility in which a departure board is installed based on an image captured by an imaging means of the departure board of the public transportation facility;
a generation means for generating guidance information including at least information on departure times after the departure time indicated on the departure board at the specified public transportation facility;
an output means for outputting the generated guidance information;
In order to configure an information processing system having
An information processing device comprising at least one of the above means. A step in which an identification unit identifies a facility of the public transportation facility in which a departure sign of the public transportation facility is installed based on an image captured by an imaging unit of the departure sign of the public transportation facility;
A generating step of generating guidance information including at least information on departure times after the departure time indicated on the departure sign at the identified facility of the public transportation facility;
an output means for outputting the generated guidance information;
An information processing method comprising: