JP7490203B2 - Mobile terminal information notification system, notification information providing system, and program. - Google Patents

Description

The present invention relates to a technology for notifying a user of a mobile terminal of information based on audio information collected by the mobile terminal.

There is a service available that superimposes advertising information onto the audio of broadcasts or the audio from speakers in commercial facilities and transmits it. With this service, the user's mobile device collects the audio with the superimposed information, extracts the information from the audio, and notifies the user of the advertisement based on this information.

For example, Japanese Patent Application Laid-Open Publication No. 2015-005069 (Patent Document 1) discloses a distribution device that receives audio information identifying audio from a terminal device that picks up the audio transmitted from digital signage, and distributes related content extracted based on the audio information to the terminal device. In addition, Japanese Patent Application Laid-Open Publication No. 2018-182585 (Patent Document 2) discloses a system that uses ultrasound to transmit information for constructing a coupon, superimposed on the provision of content including audio, such as television broadcasts or radio broadcasts.

JP 2015-005069 A JP 2018-182585 A

In recent years, multiple applications are installed on users' mobile devices, such as smartphones, as needed. Various services are provided by the applications. Such applications may be provided by operating entities, such as companies, for the purpose of advertising, for example. The inventors have considered a system that enables applications installed on such mobile devices to be linked to an information notification function, such as advertising, that uses audio information extracted from audio collected by the mobile device.

Users install and uninstall applications on their mobile devices as needed. As a result, applications on mobile devices are not constant but change. The above conventional technology did not have a mechanism for efficiently applying a notification function using voice information to applications on mobile devices that change in this way.

The present invention provides a system and program that enables efficient notification of information based on voice information generated from voice detected by a mobile terminal to multiple applications installed on the mobile terminal.

A mobile terminal information notification system according to an embodiment of the present invention includes:
a voice analysis unit that processes a voice detected by a voice detector of the mobile terminal and generates voice information based on the voice;
a linking unit that associates a notification target ID, which is assigned to the mobile device and identifies a target to which information is to be notified, with application/user information that is installed on the mobile device and includes application data indicating a plurality of applications that use the voice information or application user data indicating application users of each of the plurality of applications, and stores the associated application information in an internal or external storage device of the mobile device as application information associated data;
The method includes: acquiring notification information determined based on the voice information generated by the voice analysis unit as notification information for one of the plurality of applications or an application user indicated by the application information associated with the notification target ID in the application information correspondence data, and outputting the notification information to the mobile terminal.

According to the present disclosure, it is possible to efficiently notify multiple applications installed on a mobile device of information based on audio information generated from audio detected by the mobile device.

FIG. 1 is a diagram showing an example of the overall configuration of a system according to the first embodiment. FIG. 2 is a sequence diagram showing an example of the operation of the entire system shown in FIG. FIG. 3 is a diagram showing a detailed example of the process in S1 of FIG. FIG. 4 is a diagram showing a detailed example of the processes shown in S2 to S5 in FIG. FIG. 5 is a diagram showing a detailed example of the processes shown in S6 to S9 in FIG. FIG. 6 is a diagram showing an example of the overall configuration of a system according to the second embodiment. FIG. 7 is a diagram showing an example of the overall configuration of a system according to the third embodiment.

A mobile terminal information notification system according to an embodiment of the present invention includes:
a voice analysis unit that processes a voice detected by a voice detector of the mobile terminal and generates voice information based on the voice;
a linking unit that associates a notification target ID, which is assigned to the mobile device and identifies a target to which information is to be notified, with application/user information including application data indicating a plurality of applications that use the voice information and user data indicating application users of each of the plurality of applications, and stores the associated application information in an internal or external storage device of the mobile device as application information associated data;
and a notification unit that acquires notification information determined based on the voice information generated by the voice analysis unit as notification information for one of the multiple applications or an application user indicated by the application/user information associated with the notification target ID in the terminal application information correspondence data, and outputs the notification information to the mobile terminal.

In the above system, the app information correspondence data provided by the linking unit links information on multiple applications installed on the mobile terminal with the notification target ID assigned to the mobile terminal. When notification information determined from voice information detected on the mobile terminal is notification information for one application, the app information correspondence data can identify which application or app user among the multiple applications installed on the mobile terminal is the target of the notification. In other words, the app information correspondence data can identify the application or app user on the mobile terminal to which the notification information is targeted. The notification unit obtains notification information targeted at one application or its app user among the multiple applications of the mobile terminal indicated by the app information correspondence data, and outputs it to the mobile terminal. This makes it possible to efficiently provide information notification based on voice information generated from voice detected by the mobile terminal to multiple applications installed on the mobile terminal.

The linking unit may add app/user information for multiple applications on the mobile terminal to the app information association data at different times for each application. For example, when an application becomes available on the mobile terminal, such as when the application is installed or started, the linking unit may add the app/user information for that application to the app information data. Note that the app/user information for one application may include at least one of information indicating the application (e.g., an app identifier) or information indicating the application user (e.g., an app user identifier).

The application information correspondence data may be, for example, data indicating the correspondence between a mobile terminal and multiple applications on the mobile terminal that use notifications based on voice information. The application information correspondence data may include application/user information for multiple applications that correspond to each of the multiple mobile terminals.

In the above configuration, the notification information is determined from the voice information. The process of determining the notification information from the voice information may be executed on the mobile terminal, or may be executed outside the mobile terminal, for example, on a server. The process of determining the notification information from the voice information uses, for example, pre-stored data indicating the correspondence between the voice information and the notification information. Note that both the notification information and the notification application may be determined from the voice information. For this determination, for example, data indicating the correspondence between the voice information, the notification information, and the notification application may be used.

The mobile device information notification system may further include a coordination control unit that is installed on the mobile device and controls the voice analysis unit in accordance with the operating state of the multiple applications that use the voice information. This allows the generation of voice information to be coordinated with the operation of the multiple applications. In other words, notification of information based on voice information generated from voice detected by the mobile device can be coordinated with the multiple applications installed on the mobile device.

In the above mobile terminal information notification system, the linking unit may associate the application/user information with the notification target ID and provide the information to a server, thereby causing the application information association data to be stored in a storage device of the server. In this case, the voice analysis unit may transmit the voice information to the server in association with the notification target ID. The notification unit may obtain the notification information from the server.

With this configuration, the server determines notification information based on the voice information received from the mobile device, and further determines the application or app user to which the notification information is directed based on the notification target ID and app information corresponding data received from the mobile device. This allows the notification unit of the mobile device to obtain notification information for the application or app user indicated by the app/user information in the app information corresponding data from the server.

A notification information providing system according to an embodiment of the present invention includes:
a voice information receiving unit that receives voice information generated based on a voice detected by a detector of the mobile terminal, in association with a notification target ID that is assigned to the mobile terminal and identifies a target to which information is to be notified;
an application information correspondence data storage unit configured to store application information correspondence data that associates the notification target ID assigned to the mobile terminal with application/user information including application data indicating a plurality of applications that are installed on the mobile terminal and use the voice information or application user data indicating application users of each of the plurality of applications;
a notification information data storage unit that stores the voice information, notification information corresponding to the voice information, and notification information data indicating a notification application;
a notification information determination unit that determines, using the notification information data, notification information and a notification application corresponding to the voice information received by the voice information receiving unit;
a notification destination determination unit that uses the application information correspondence data to determine, as a notification destination, an application or an application user that corresponds to both the notification application determined by the notification information determination unit and the notification target ID received by the voice information receiving unit;
The notification information supply unit supplies the notification information to the notification destination determined by the notification destination determination unit.

In the above configuration, the notification information providing system holds application information correspondence data that associates application/user information of multiple applications that use voice information on a mobile terminal with a notification target ID. As a result, the application/user information of multiple applications installed on the mobile terminal is linked to the notification target ID. As a result, when the notification information providing system receives voice information and a notification target ID from a mobile terminal, it can determine notification information and a notification application from the voice information, and further specify an application installed on the mobile terminal or its application user as a notification destination. Therefore, the notification information providing system can determine notification information and a notification destination based on the voice information and transmit them to an appropriate mobile terminal. In other words, the notification information providing system can grasp multiple applications on a mobile terminal and supply notification information determined from voice information received from the mobile terminal to an appropriate notification destination. As a result, it becomes possible to efficiently perform information notification based on voice information extracted from voice collected by the mobile terminal to multiple applications installed on the mobile terminal.

The following describes the embodiments with reference to the drawings. The same or corresponding components in the drawings are given the same reference numerals, and the same description will not be repeated. Note that, in order to make the description easier to understand, the drawings referred to below show simplified or schematic configurations, and some components are omitted.

[Embodiment 1]
(System configuration example)
FIG. 1 is a diagram showing an example of the overall configuration of a system in the first embodiment. In the example shown in FIG. 1, a mobile terminal information notification system 2 is implemented in a mobile terminal 1. A notification information providing system 3 is constructed in a server. The mobile terminal 1 is capable of data communication with the server. Hereinafter, the notification information providing system 3 may be referred to as a server 3. The mobile terminal information notification system 2 extracts voice information from a voice detected in the mobile terminal 1, transmits it to the server 3, and receives notification information from the server 3 in response. The mobile terminal 1 outputs the notification information to a user. The server 3 receives the voice information from the mobile terminal 1, and transmits notification information based on the voice information to the mobile terminal 1. In this embodiment, for a plurality of applications installed in the mobile terminal 1, notification of information based on voice information generated from a voice detected in the mobile terminal 1 is performed.

(Example of the configuration of a mobile terminal information notification system)
The mobile terminal information notification system 2 includes a voice analysis unit 21, a notification unit 22, a cooperation control unit 23, and a linking unit 24. The voice analysis unit 21 processes the voice detected by a voice detector 10 (hereinafter referred to as the detector 10) of the mobile terminal 1, and extracts voice information contained in the voice. The detector 10 is, for example, a microphone provided in the mobile terminal 1. Alternatively, the detector 10 may detect the voice reproduced by the mobile terminal 1. The detector 10 detects the voice reproduced by the voice reproducer 4. The voice reproducer 4 is not particularly limited. For example, the detector 10 detects the voice of a broadcast wave receiving and reproducing device such as a television or radio, or the voice distributed via the Internet and reproduced by the mobile terminal 1.

As an example, the voice analysis unit 21 may generate a voice fingerprint from the detected voice. In this case, the voice fingerprint becomes the voice information. The voice fingerprint is data indicating the characteristics of the voice generated from the voice signal using a specific algorithm. The voice signal is converted into a character string called a plurality of hash values (one example of a voice fingerprint) by a specific algorithm. The voice can be uniquely identified by the consistency of the hash values. The same voice fingerprint is generated from the same voice signal. The data size of the voice fingerprint is smaller than that of the original voice. Note that the voice information may be the voice fingerprint (hash value) itself, or an identifier based on the voice fingerprint. For example, an identifier that uniquely corresponds to the voice fingerprint (hash value) may be the voice information.

The voice analysis unit 21 may provide the voice information generated by the voice analysis unit 21 to, for example, the server 3. As an example, the voice analysis unit 21 may transmit the voice information to the server 3 in association with a notification target ID that identifies a target to which the information is to be notified. One notification target ID is assigned to one mobile terminal. The notification target ID can also be said to identify the user of the mobile terminal 1. The notification target ID may be, for example, an advertisement ID for identifying a target of an advertisement. Examples of advertisement IDs include IDFA (Identifier for Advertisers) or GAID (Google Play Services ID for Android) (GOOGLE and ANDROID are registered trademarks). Alternatively, the notification target ID may be an identifier that is assigned to each mobile terminal and managed by the server 3 or other external device. The voice analysis unit 21 may transmit other information of the notification target ID to the server 3 in association with the voice information. The mobile terminal 1 receives notification information based on the voice information and the notification target ID from the server 3. The notification information is information that should be notified to the user of the application installed in the mobile terminal 1. The notification unit 22 acquires the notification information and outputs it to the mobile device.

The manner in which the mobile terminal 1 receives notification information from the server 3 includes a case in which at least one of a plurality of applications receives the notification information. As an example, when the notification information is a push notification, the OS of the mobile terminal receives the notification information and can output the push notification to the mobile terminal 1 via the application. In this case, the push notification arrives at the mobile terminal 1 from the server 3 via the application server (app server). Note that the notification information may be sent to the mobile terminal 1 via another server, or may be output without going via the application.

The notification unit 22 acquires notification information based on the voice information in the mobile terminal 1 and outputs it to the mobile terminal 1. The notification unit 22 outputs the notification information to an output device 11 provided in the mobile terminal 1, such as a display or a speaker. The notification unit 22 receives, for example, notification information for an application installed in the mobile terminal 1 or for a user of the application. As an example, the notification unit 22 may be implemented as part of the application or the OS of the mobile terminal 1. The notification information received by the notification unit 22 may be, for example, information received by the application or OS from an app server, server 3, or a server that supplies other notification information.

The cooperation control unit 23 controls the voice analysis unit 21 according to the operating states of multiple applications installed in the mobile terminal 1. The cooperation control unit 23 may, for example, record information indicating an application that has requested the voice analysis unit 21 to start in the mobile terminal 1, and control the voice analysis unit 21 according to the operating state of that application. For example, after the voice analysis unit 21 is started in response to a start request from one application (e.g., app A), if another application (e.g., app B) requests start during the start of the voice analysis unit 21, the cooperation control unit 23 records these two applications (app A and app B) as multiple applications that use voice information. The cooperation control unit 23 can control the voice analysis unit 21 according to the operating states of these multiple applications.

As an example, when the voice analysis unit 21 is started by a start request from one application, and at least one other application makes a start request while the voice analysis unit 21 is running, if either the one application or one of the other applications is in an active state, the cooperation control unit 23 can control the voice analysis unit 21 to maintain its running state, and if all of these applications become inactive, to stop the processing of the voice analysis unit 21. This allows the functions of the voice analysis unit 21 to be effectively utilized by multiple applications installed on the mobile terminal 1.

In the mobile terminal 1, an application that uses voice information, i.e., an application that cooperates with notification of information based on voice information, may be registered in an external device such as the mobile terminal 1 or the server 3. The registration is performed, for example, by recording information that identifies the application, such as an app ID, in a storage unit accessible from the mobile terminal 1 or the server 3. The registration may be performed automatically, for example, when the application is installed, or may be performed in response to a user operation on the mobile terminal 1.

The collaboration control unit 23 may detect, for example, an active state/inactive state of an application as the operating state of the application. The active state/inactive state may be a running/stopped state of the application, a foreground execution state/background execution state, or another operating mode state. In addition to the active state/inactive state, for example, a collaboration-enabled state/a collaboration-disabled state, etc. may be detected as the operating state of the application.

The cooperation control unit 23 may control the execution/stop of the processing of the voice analysis unit 21 according to the operating states of multiple applications that are installed in the mobile terminal 1 and use the voice information of the voice analysis unit 21. As a result, at least one of the processing of generating voice information by the voice analysis unit 21 or the processing of providing voice information to the server 3 is controlled according to the operating states of the multiple applications. For example, the timing of the processing of the voice analysis unit to process voice and generate voice information can be controlled according to the operating states of the multiple applications installed in the mobile terminal 1. As an example, it is possible to control so that the processing of generating voice information by the voice analysis unit 21 is enabled when any one of the multiple applications is in an active state. As a result, the operation of the voice analysis unit 21 can be stopped when all of the multiple applications are inactive. As a result, the processing load on the mobile terminal 1 can be reduced.

The cooperation control unit 23 may control the voice analysis unit 21 according to the operation state of each of the multiple applications. For example, the voice analysis unit 21 may be controlled to transmit information indicating the operation state of the application together with the voice information to the server 3. Also, for example, the voice analysis unit 21 may be controlled to transmit to the server 3 the voice information of an application that permits notification of information based on voice information, and not to transmit to the server 3 the voice information of an application that does not permit such notification. In this case, the voice analysis unit 21 may determine which application the generated voice information is related to by using pre-recorded correspondence data. The correspondence data is data indicating the correspondence between the voice information and the application. For example, the correspondence data may be provided from the server 3.

As another example, the cooperation control unit 23 may control the voice analysis unit 21 to send voice information related to any of a plurality of applications installed on the mobile terminal 1 that use voice information from the voice analysis unit 21 to the server 3, and not to send voice information not related to any of the applications to the server 3. This allows efficient notification based on voice information about applications installed on the mobile terminal 1. Note that, as a variation of this, the cooperation control unit 23 can control the voice analysis unit 21 to send voice information related to an active application out of the plurality of applications to the server 3, and not to send voice information of an inactive application to the server 3.

The cooperation control unit 23 may control the voice analysis unit 21 based on information provided from the application. The cooperation control unit 23 may, for example, control the voice analysis unit 21 so that information regarding at least one of the multiple applications is associated with the voice information and transmitted to the server 3. This allows information regarding the applications installed on the mobile terminal 1 to be provided to the server 3 together with the voice information. Therefore, the server 3 can provide notification information to the mobile terminal 1 taking into consideration the applications installed on the mobile terminal 1. The information regarding the application may include, for example, in addition to the app ID, the user ID of the application (app user ID), information regarding the user of other applications, or information input by the user in the application.

The linking unit 24 associates the notification target ID with an app ID that identifies each of the multiple applications or an app user ID that identifies each of the multiple applications, and transmits the associated app ID to the server 3. As a result, the notification target ID is associated with the app/user information and stored as app information association data in the storage unit 36 of the server 3. That is, the linking unit 24 transmits the app IDs or app user IDs of the multiple applications that are installed on the mobile terminal 1 and use the voice information of the voice analysis unit 21, that is, the app/user information, together with the notification target ID to the server 3. As a result, the app information association data is stored in the storage unit 36 of the server 3. Note that the app/user information may include both the app ID and the app user ID, or may include either one of them.

The timing at which the linking unit 24 transmits the application/user information to the server 3 is not particularly limited. In this embodiment, the linking unit 24 transmits the application/user information to the server 3 separately from the process in which the voice analysis unit 21 transmits the voice information to the server 3. For example, when an application is installed on the mobile terminal 1, the linking unit 24 may transmit the application/user information of the application together with the notification target ID to the server 3. Alternatively, when the application is started or when the application becomes available on the mobile terminal 1, the application/user information may be associated with the notification target ID and transmitted to the server 3. Note that the application information/user information may be associated with the voice information and transmitted to the server 3 when the voice analysis unit 21 transmits the voice information to the server 3. In this case, the linking unit 24 functions as a part of the voice analysis unit 21.

The linking unit 24 may provide the server 3 with an application user ID in addition to the application ID as application/user information in association with the notification target ID and store it as application information association data. The application user ID is set for each application. The application user IDs of multiple applications may be different from each other. The application user IDs of each of the multiple applications are provided to the server 3 in association with the notification target ID. This allows the server 3 to recognize that these application user IDs belong to the same user, even if the application user IDs are different for each application. This makes it possible to notify information across multiple applications and collect user information.

The linking unit 24 stores application information correspondence data in the storage unit 36 of the server 3, etc., which associates the notification target ID assigned to the mobile terminal 1 with the application/user information of multiple applications installed on the mobile terminal 1 and using voice information. The voice information generated by the voice analysis unit 21 of the mobile terminal 1 is sent to the server 3 together with the notification target ID, for example. The server 3 determines notification information based on this voice information. Furthermore, the application or application user installed on the mobile terminal 1 is determined as the notification destination of this notification information. This determination can use the application information correspondence data in the storage unit 36 provided by the linking unit 24. Notification information with the determined application or application user as the notification destination is sent from the server 3 to the mobile terminal 1. The notification unit 22 of the mobile terminal 1 receives notification information with the application or application user determined using the application information correspondence data as the notification destination. That is, the notification unit 22 of the mobile terminal 1 receives notification information with the application of the mobile terminal 1 or its application user as the notification destination, which is indicated by the application information correspondence data. The notification unit 22 has a function to receive notification information addressed to an application of the mobile terminal 1 or to a user of the application, and a function to output the notification information to the mobile terminal 1.

The mobile device information notification system 2 can be configured by a computer provided in the mobile device 1. The computer has a processor and a memory. The processor executes processing according to a program recorded in the memory, thereby realizing the functions of the voice analysis unit 21, the notification unit 22, the cooperation control unit 23, and the linking unit 24. A program for causing a computer to function as a mobile device information notification system, or a non-transitory recording medium on which the program is recorded, is also included in the embodiments of the present invention. Note that the mobile device information notification system 2 may be configured by two or more computers.

(Example of the configuration of the notification information providing system (server))
The notification information providing system 3 (server 3) includes a voice information receiving unit 31, a notification information determining unit 32, a notification destination determining unit 33, a notification information supplying unit 34, a voice information registering unit 35, and a storage unit 36. The voice information receiving unit 31 receives voice information generated based on the voice detected by the detector 10 of the mobile terminal 1, in association with the notification target ID. In other words, the voice information receiving unit 31 receives the voice information and the notification target ID transmitted from the voice analyzing unit 21 of the mobile terminal information notification system 2.

The storage unit 36 stores application information correspondence data and notification information data. The application information correspondence data is data that associates a notification target ID assigned to the mobile terminal 1 with application/user information of multiple applications installed in the mobile terminal 1 and using voice information. The application information correspondence data may include application/user information corresponding to each notification target ID for multiple notification target IDs. The application/user information includes at least one of application data (e.g., application ID) indicating an application or application user data (e.g., application user ID) indicating an application user of the application. The notification information providing system 3 receives the notification target ID and application/user information from the linking unit 24 of each of the multiple mobile terminals, and records the application information correspondence data in the storage unit 36 as application information correspondence data. For example, the application user ID of each of multiple applications in one mobile terminal and the notification target ID assigned to the mobile terminal are associated with each other by the application information correspondence data. This makes it possible to grasp the users of each application of the mobile terminal as the same user. Note that the application information correspondence data may include both the application ID and the application user ID corresponding to the notification target ID, or may include only the application ID.

The memory unit 36 stores notification information data. The notification information data includes voice information, and information indicating notification information and a notification application corresponding to the voice information. In other words, the notification information data is data indicating the correspondence between the voice information, and the notification information and a notification application. The notification information data includes, for multiple pieces of voice information, information indicating notification information and a notification application corresponding to each of the multiple pieces of voice information. The notification information data is stored in advance in the memory unit 36. For example, the voice information registration unit 35 can accept input of notification information data from an administrator and record it in the memory unit 36.

The notification information determination unit 32 uses the notification information data to determine the notification information and notification application corresponding to the voice information received by the voice information receiving unit 31. In other words, the notification information determination unit 32 determines the notification information based on the voice information.

The notification destination determination unit 33 uses the app information correspondence data in the memory unit 36 to determine, as the notification destination, an application or app user corresponding to both the notification application determined by the notification information determination unit 32 and the notification target ID received by the audio information receiving unit 31. For example, the app information correspondence data may include an app ID and an app user ID corresponding to the notification target ID. In this case, the notification destination determination unit 33 refers to the app information correspondence data to obtain an app user ID corresponding to both the app ID of the notification application determined by the notification information determination unit 32 and the notification target ID received by the audio information receiving unit 31. The obtained app user ID is determined as the notification destination.

The notification information determination unit 32 and the notification destination determination unit 33 are not limited to the above example. For example, the notification information determination unit 32 may use the received notification target ID and the notification information data and application information correspondence data in the storage unit 36 in determining the notification information. For example, the notification information determination unit 32 uses the application information correspondence data to acquire an application ID corresponding to the received notification target ID. The notification information determination unit 32 can determine the notification information to be transmitted that corresponds to the acquired application ID and also corresponds to the received voice information. That is, the notification information associated with the received voice information and the acquired application ID can be extracted from the multiple notification information recorded as application notification data. In this case, the notification information determination unit 32 can also determine the application to which the notification information is to be transmitted. Therefore, the notification information determination unit 32 also serves as the notification destination determination unit 33.

In this way, the notification information determination unit 32 refers to the correspondence between the notification target ID and the application/user information received from the linking unit 24 of the mobile terminal 1 and recorded in the memory unit 36. This makes it possible to obtain the application/user information corresponding to the voice information received by the voice information receiving unit 31. In other words, it is possible to identify the application installed on the mobile terminal 1 that is the notification target indicated by the notification target ID. This makes it possible for the notification information determination unit 32 to determine the notification information for the application installed on the mobile terminal 1 as the notification information to be sent to the mobile terminal 1.

The notification information supply unit 34 transmits the notification information determined by the notification information determination unit 32 directly or indirectly to the mobile terminal 1. When the notification information supply unit 34 transmits the notification information directly to the mobile terminal 1, the notification information may be transmitted in association with, for example, data related to the application to be notified (e.g., authentication data for accessing the application). When the notification information supply unit 34 transmits the notification information indirectly to the mobile terminal 1, the notification information may be transmitted in association with, for example, an application user ID, to an application server that manages the application of the application ID. In this case, the application server can identify the destination mobile terminal 1 based on the application user ID and transmit the notification information to the mobile terminal 1.

The voice information registration unit 35 acquires voice information and notification information, associates the voice information and notification information with each other, and records them in the storage unit 36 as notification information data. The voice information registration unit 35 can accept input of voice information and notification information, for example, from an administrator. In addition to the voice information and notification information, the voice information registration unit 35 may also accept input of other related information from the administrator, associate it with the voice information or notification information, include it in the notification information data, and record it in the storage unit 36.

The audio information registration unit 35 may obtain an audio file as input for audio information, and process the audio file to generate audio information. For example, an audio fingerprint (hash value) may be generated from the audio file, and this may be used as the audio information. Alternatively, an identifier (ID) that uniquely corresponds to the audio fingerprint (hash value) may be used as the audio information. In this case, the algorithm used to generate the audio information is the same as the algorithm used when the audio analysis unit 21 in the mobile terminal 1 generates an audio fingerprint from audio. In other words, the audio fingerprint generated from the audio file in the server 3 and the audio fingerprint generated in the mobile terminal 1 from the same audio as the audio in the audio file will be the same.

The voice information registration unit 35 may record the voice information and notification information in association with an app ID, which is an identifier of the application that provides the notification information to the user. In this case, the voice information registration unit 35 accepts input of information indicating the application from the administrator, in addition to the voice information and notification information. As a result, the voice information, notification information, and app ID are recorded in association with each other as notification information data in the memory unit 36.

The voice information registration unit 35 may also set an upper limit on the number of times notification information is provided for voice information of voice detected by a mobile terminal during a specified period (an upper limit on the number of times notification information is provided for the same voice information to the same mobile terminal during a specified period). In this case, an upper limit number and a specified period are set. As the upper limit number, for example, a frequency cap number is set. As the specified period, for example, a frequency cap type (day, week, month) is set.

The notification information providing system 3 can be configured with one or more computers (server computers). The computer has a processor and a memory. The processor executes processing according to a program recorded in the memory, thereby realizing the functions of the voice information receiving unit 31, the notification destination determining unit 33, the notification information determining unit 32, the notification information supplying unit 34, and the voice information registering unit 35. A program for causing a computer to function as a mobile terminal information notification system, or a non-transitory recording medium on which the program is recorded, is also included in an embodiment of the present invention.

(Example of operation)
FIG. 2 is a sequence diagram showing an example of the operation of the entire system shown in FIG. 1. In the example shown in FIG. 2, the voice information registration unit 35 of the server 3 records voice information, notification information, and an application ID in association with each other in the storage unit 36 (S1). FIG. 3 is a diagram showing a detailed example of the process in S1. In the example shown in FIG. 3, the voice information registration unit 35 accepts input from the administrator of the provision of a voice file, notification information, application information indicating the application of the notification information, a campaign ID, an upper limit number, and a period for setting the upper limit number. For example, the voice file is uploaded to the server 3 by the operation of the administrator. The voice information registration unit 35 generates a voice fingerprint (hash value) for the voice of the voice file. The voice information registration unit 35 generates a voice information ID based on the generated voice fingerprint. For example, the character string of the voice fingerprint (hash value) may be used as the voice information ID as it is, or the voice fingerprint may be processed by adding further characters to the character string of the voice fingerprint, and the voice information ID may be obtained by performing a predetermined processing on the voice fingerprint. Alternatively, the identifier of the voice file corresponding to the voice fingerprint (hash value) may be used as the voice information ID. In this case, data indicating the correspondence between the audio fingerprint (hash value) and the audio information ID may be recorded in the storage unit 36. The audio information ID is an example of audio information.

In this example, a case will be described in which the operator of an application installed on the mobile terminal 1 runs a sales promotion campaign. In this case, the notification information includes information about the campaign. The audio file is an audio file of audio included in content provided to the public during the campaign period, for example, by broadcasting on radio or television or by distribution via the Internet. The campaign ID is an identifier for the campaign. A notification information ID is assigned to the notification information.

The audio information registration unit 35 associates the audio information ID, application ID, and notification information, and records them in the storage unit 36 as application notification data. In the example shown in FIG. 3, these data are recorded by forming an audio-notification correspondence table and a notification information table. Note that the data format is not limited to the example shown in FIG. 3. The audio-notification correspondence table records the audio information ID, application ID, notification information ID, campaign ID, upper limit number, and period for setting the upper limit number in association with each other. The notification information table records the notification information ID, title, text, and other information related to notifications in association with each other. As an example, the notification information is information on a push notification output to a user by an application on the mobile terminal 1.

2, when application A (app A) is started on the mobile terminal 1, the cooperation control unit 23 starts the voice analysis unit 21 and starts the cooperation operation (S2). The linking unit 24 transmits the application ID, application user ID, and notification target ID of application A to the server 3 (S3). The server 3 records the application ID, application user ID, and notification target ID in the storage unit 36 as application information data in association with each other (S3). Next, when application B (app B) is started, the cooperation control unit 23 causes the voice analysis unit 21 to start the cooperation operation (S4). At this time, since the voice analysis unit 21 has already started, the cooperation control unit 23 issues a cooperation instruction without issuing a start instruction. The linking unit 24 transmits the application ID, application user ID, and notification target ID of application B to the server 3 and stores them in the storage unit 36 (S5). Here, both applications A and B are one of a plurality of applications recognized as applications that use voice information in the mobile terminal 1. Note that, although the processes of S2 to S5 are triggered by the start of an application, the execution timing is not limited to this. For example, the trigger may be when the application is installed, when the state switches from an inactive state to an active state, or when the state switches from a notification disallowed state to a notification allowed state.

Figure 4 is a diagram showing a detailed example of the processes shown in S2 to S5. In the example of Figure 4, the linking unit 24 associates the application ID, notification target ID, and application user ID for each of applications A and B launched on the mobile terminal 1, and transmits them to the server 3. The server 3 associates these and records them in the storage unit 36 as application information corresponding data. In the example of Figure 4, these data are recorded as application information corresponding data in the format of a terminal application table and an application information table. Note that the data format is not limited to the example shown in Figure 4. In the terminal application table, the notification target ID, application ID, and application user ID are recorded in association with each other. In the application information table, the application ID and data transmission destination (e.g., callback URL) are recorded in association with each other.

Referring to FIG. 2, the audio player 4 plays back the content including the audio that is the basis of the audio information registered in S1. The audio analysis unit 21 generates audio information from the audio detected by the detector 10 (S6). The audio analysis unit 21 transmits the generated audio information to the audio information receiving unit 31 of the server 3 in association with the notification target ID (S7). The audio information receiving unit 31 passes the audio information and the notification target ID to the notification information determination unit 32. The notification information determination unit 32 acquires notification information corresponding to the received audio information and the corresponding application ID from the notification information data in the storage unit 36 (S8). The notification destination determination unit 33 acquires the application ID and the application user ID corresponding to the notification target ID from the application information correspondence data (S9). The notification information supply unit 34 receives the notification information from the notification information determination unit 32, receives the application user ID as the notification destination from the notification destination determination unit 33, and supplies them to the application server. The application server sends notification information to the user of application A on mobile device 1 (the user indicated by the application user ID) (S10).

5 is a diagram showing a detailed example of the processing of S6 to S10 in FIG. 2. In the example of FIG. 5, the voice analysis unit 21 processes the voice detected by the detector 10 to generate a voice fingerprint. In this example, the voice fingerprint is output as a hash value, and the hash value is linked to the voice information ID that manages the voice file. A table showing the correspondence between the hash value and the voice information ID is recorded in advance in the storage unit 36. The algorithms for generating the voice fingerprint and the hash value are the same as those in the voice information registration unit 35 of the server 3. That is, the same hash value is generated from the same voice in the mobile terminal 1 and the server 3. The voice analysis unit 21 transmits the voice information including the hash value and the notification target ID to the server 3 in association with each other. The voice analysis unit 21 may further transmit an app ID indicating the linked application to the server 3 in association with the notification target ID. For example, the app ID of the linked application (app A and app B in this example) in the mobile terminal 1 may be transmitted to the server 3 as the app ID.

These data are acquired by the notification information determination unit 32 via the voice information receiving unit 31 of the server 3. The notification information determination unit 32 can identify the voice information ID corresponding to the voice information received by the voice information receiving unit 31 by using data indicating the correspondence between the voice information and the voice information ID, which is pre-recorded in the storage unit 36. The notification information determination unit 32 acquires the application ID and notification information ID corresponding to the voice information ID from the voice-notification correspondence table in FIG. 3. The notification destination determination unit 33 acquires the application user ID corresponding to both the application ID and the notification target ID acquired by the notification information determination unit 32 from the terminal application table shown in FIG. 4. This acquires the application user ID of the notification application that is an application installed on the mobile terminal 1 and is the target of the notification information. Furthermore, the notification information determination unit 32 acquires the title and the body corresponding to the acquired notification information ID from the notification information table. This acquires the title and the body as the notification information corresponding to the voice information ID.

As shown in FIG. 5, the notification information determination unit 32 passes the title and text of the acquired notification information, the application user ID, the campaign ID, and the data transmission destination to the notification information supply unit 34. The notification information supply unit 34 transmits the title, text, user ID, and campaign ID to the application server 5 (application server of application A) indicated by the data transmission destination. The application server 5 transmits PUSH notification information including the title and text to application A of the mobile terminal 1. Application A outputs the title and text on the mobile terminal 1 by the PUSH notification.

According to this embodiment, as shown in Figs. 1 and 5, notification information based on voice information generated by the voice analysis unit 21 can be appropriately provided for multiple apps A and B in the mobile terminal 1. This allows the multiple apps A and B to share the function of receiving notification information based on voice information. In addition, it becomes possible to control the notification function according to the operating states of apps A and B. For example, if at least one of the multiple registered apps A and B is in an active state, when voice information is generated for an inactive app, notification information can be provided to this inactive app B.

In addition, in this embodiment, in the terminal application table shown in FIG. 4, application user IDs "USR_AAA" and "USR_BBB" are recorded in association with the same notification target ID "A0001". This links the user of application A's user ID "USR_AAA" and the user of application B's user ID "USR_BBB" to the same person. This can be used to notify information across multiple services provided by multiple applications, or to measure the effectiveness, for example.

[Embodiment 2]
FIG. 6 is a diagram showing an example of the overall system configuration in the second embodiment. In the system configuration shown in FIG. 6, the mobile terminal 1 acquires notification information data from the server 3 and stores it in the storage unit 12 of the mobile terminal 1. The server 3 may also use the application information correspondence data to acquire application IDs of multiple applications installed in the mobile terminal 1, extract notification information corresponding to the multiple application IDs installed in the terminal from the notification information data in the storage unit 36, and transmit the extracted notification information to the mobile terminal 1. This allows the mobile terminal 1 to efficiently acquire data indicating the correspondence between the notification information and the voice information of multiple applications installed in the mobile terminal 1 and using voice information. For example, as in the first embodiment, the linking unit 24 provides the server 3 with application information correspondence data including the application/user information and the notification target ID of the application installed in the mobile terminal 1. The server 3 provides the notification information of the application indicated by the application/user information and the notification information data including the voice information corresponding to the notification information to the mobile terminal 1 specified by the notification target ID. Note that the form of providing the notification information data from the server 3 to the mobile terminal 1 is not limited to this. For example, regardless of the application of the mobile terminal 1, notification information data including all notification information or notification information selected based on other criteria may be supplied to the mobile terminal 1.

When notification information corresponding to the voice information generated from the detected voice is present in the notification information data in the memory unit 12, the voice analysis unit 21 transmits the notification information ID, the notification target ID, and the application ID to the server 3. The notification information determination unit 32 of the server 3 determines the notification information based on the received notification information ID, and the notification destination determination unit 33 determines the application user ID as the notification destination based on the notification target ID or the application ID. The notification information supply unit 34 supplies the notification information and the application user ID to the application server, and the notification information is transmitted from the application server to the application of the mobile terminal 1.

[Embodiment 3]
FIG. 7 is a diagram showing an example of the overall system configuration in the third embodiment. In the example shown in FIG. 7, the notification information determination unit 32 and the notification destination determination unit 33 are not in the server 3 but in the mobile terminal 1. Therefore, the notification information data and the application information corresponding data are stored in the storage unit 12 of the mobile terminal 1. As for the notification information data, the notification information data is supplied from the server 3 to the mobile terminal 1 in the same manner as in the case of FIG. 6. As for the application information corresponding data, the linking unit 24 acquires information on the application of the mobile terminal 1, i.e., application/user information, and records it in the storage unit 12 as application information corresponding data. The processing of the notification information determination unit 32 and the notification destination determination unit 33 can be executed in the same manner as in the first embodiment. Note that, in the configuration of FIG. 7, as shown by the dashed line, the linking unit 24 may store the application/user information and the notification target ID in the storage unit 36 of the server 3 as application information corresponding data. In this case, the mobile terminal 1 acquires the application information data from the server 3, stores it in the storage unit 12 of the mobile terminal 1, and can be made accessible from the notification destination determination unit 33. As a result, the application information corresponding data is managed in the server 3. Therefore, for example, even if the data stored in the mobile terminal 1 is deleted or changed, the information in the application information corresponding data can be maintained.

(Other Modifications)
The audio information extracted from the audio by the audio analysis unit is not limited to the audio fingerprint. For example, the audio information may be decoded information obtained by decoding information superimposed on the audio signal. Furthermore, the audio to be processed by the audio analysis unit is not limited to audio in the audible range.

The manner in which the voice information receiving unit 31 receives data from the mobile terminal 1 includes receiving data directly from the mobile terminal 1, as well as receiving data indirectly, such as receiving data from another server that has been sent from the mobile terminal 1.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above embodiments.

1: Mobile terminal, 2: Mobile terminal information notification system, 3: Notification information providing system (server), 21: Voice analysis unit, 22: Notification unit, 23: Collaboration control unit, 24: Linking unit, 31: Voice information receiving unit, 32: Notification information determination unit, 33: Notification destination determination unit, 34: Notification information supply unit, 35: Voice information registration unit, 36: Storage unit

Claims (4)

a voice analysis unit that processes a voice detected by a voice detector of the mobile terminal and generates voice information based on the voice;
a linking unit that associates a notification target ID, which is assigned to the mobile device and identifies a target to which information is to be notified, with application/user information that is installed on the mobile device and includes application data indicating a plurality of applications that use the voice information or application user data indicating application users of each of the plurality of applications, and stores the associated application information in an internal or external storage device of the mobile device as application information associated data;
a notification unit that acquires notification information determined based on the voice information generated by the voice analysis unit as notification information for one application or an application user among the plurality of applications indicated by the application/user information associated with the notification target ID in the application information correspondence data, and outputs the notification information to the mobile device ;
A mobile terminal information notification system comprising: a coordination control unit that is installed on the mobile terminal and controls the voice analysis unit according to the operating state of the plurality of applications that use the voice information . The mobile terminal information notification system according to claim 1 ,
the association unit associates the application/user information with the notification target ID and provides the application/user information to a server, thereby causing the application information association data to be recorded in a storage device of the server;
The voice analysis unit transmits the voice information to the server in association with the notification target ID,
The notification unit acquires the notification information from the server. a voice information receiving unit that receives voice information generated based on a voice detected by a detector of the mobile terminal, in association with a notification target ID that is assigned to the mobile terminal and identifies a target to which information is to be notified;
an application information correspondence data storage unit configured to store application information correspondence data that associates the notification target ID assigned to the mobile terminal with application/user information including application data indicating a plurality of applications that are installed on the mobile terminal and use the voice information or application user data indicating application users of each of the plurality of applications;
a notification information data storage unit that stores the voice information, notification information corresponding to the voice information, and notification information data indicating a notification application;
a notification information determination unit that determines, using the notification information data, notification information and a notification application corresponding to the voice information received by the voice information receiving unit;
a notification destination determination unit that uses the application information correspondence data to determine, as a notification destination, an application or an application user that corresponds to both the notification application determined by the notification information determination unit and the notification target ID received by the voice information receiving unit;
a notification information supply unit that supplies the notification information to the notification destination determined by the notification destination determination unit;
A notification information providing system comprising: a collaborative control unit that is installed on the mobile terminal and controls a voice analysis unit that generates the voice information based on the voice detected by the detector of the mobile terminal in accordance with the operating state of the multiple applications that use the voice information. a voice analysis process for processing a voice detected by a voice detector of the mobile terminal and generating voice information based on the voice;
a linking process for linking a notification target ID, which is assigned to the mobile device and identifies a target to which information is to be notified, with application/user information including application data indicating a plurality of applications that are installed on the mobile device and use the voice information or application user data indicating application users of each of the plurality of applications, and recording the linking data as application information corresponding data in an internal or external storage device of the mobile device;
a notification process of acquiring notification information determined based on the voice information generated by the voice analysis process as notification information for one application or an application user among the plurality of applications indicated by the application/user information associated with the notification target ID in the application information correspondence data, and outputting the notification information to the mobile terminal ;
A program that causes a computer to execute a cooperative control process that is installed on the mobile terminal and controls the voice analysis process in accordance with the operating states of the plurality of applications that use the voice information .

Images