JP6896594B2 - Music selection device, music selection method and program - Google Patents

Description

The present invention relates to a music selection device, a music selection method, and a program for selecting a music that suits a user's taste.

Media content that suits the user's taste is selected and recommended to the user. Patent Document 1 describes that an emotional state of a user is input and media content corresponding to the input emotional state is recommended.

U.S. Patent Application Publication No. 2014/0298364

In the technique described in Patent Document 1, the user needs to input an emotional state in order to recommend media content. Therefore, if the user is not accustomed to inputting the emotional state, there is a problem that noise increases. Further, there is a problem that the work of inputting the emotional state is complicated for the user.

Therefore, the present invention has been made in view of these points, and provides a music selection device, a music selection method, and a program capable of recommending a music that suits the user's taste without inputting an emotional state. The purpose is to provide.

The music selection device of the first aspect of the present invention is based on a data acquisition unit that acquires user data associated with a user's social network account and the user data acquired by the data acquisition unit. An emotion estimation unit that estimates emotions and music data indicating the characteristics of a plurality of songs are acquired, and the user's emotions estimated by the emotion estimation unit are compared with the characteristics of the plurality of songs indicated by the music data. Thereby, it has a selection unit for selecting a recommended music from a plurality of music.

Further, the music selection device further has a storage unit for storing the relationship between the plurality of songs and human emotions, and the selection unit estimates the emotions based on the relationship stored in the storage unit. The recommended music suitable for the user's emotions estimated by the department may be selected.

The storage unit may memorize the relationship by memorizing the positions of the plurality of musical tunes in the emotion map showing the characteristics of the musical tunes. The storage unit may store the relationship between the plurality of songs and the user's emotions for each type of the user's emotions.

The music selection device may further include an update unit that changes the relationship stored in the storage unit based on feedback when the user hears the recommended music selected by the selection unit in the past. .. The selection unit may select a new recommended song based on the user's preference estimated based on the feedback when the user listens to the recommended song selected in the past.

The selection unit may select a new recommended song based on the degree of similarity with the recommended song selected immediately before. While searching for the preference of the user, the selection unit may select the recommended music whose similarity to the recommended music selected immediately before is smaller than a predetermined threshold value. After the search for the user's preference is completed, the selection unit may select the recommended music having a similarity with the recommended music selected immediately before and a predetermined threshold value or more.

After the search for the user's preference is completed, the selection unit further includes a detection unit that monitors changes in the user's preference based on feedback when the user listens to the recommended music selected in the past. May restart the search for the user's preference when the detection unit detects that the preference has changed.

The music selection method of the second aspect of the present invention includes a step of acquiring user data associated with a user's social network account and music data indicating the characteristics of a plurality of music, which are executed by a computer. By comparing the step of estimating the user's emotions based on the user data, the estimated user's emotions, and the characteristics of the plurality of songs indicated by the music data, a recommended music is selected from the plurality of songs. It has a step to select and.

The program of the third aspect of the present invention uses a computer as a data acquisition unit that acquires user data associated with an account of the user's social network, and the user data acquired by the data acquisition unit. An emotion estimation unit that estimates emotions and music data indicating the characteristics of a plurality of songs are acquired, and the user's emotions estimated by the emotion estimation unit are compared with the characteristics of the plurality of songs indicated by the music data. By doing so, it functions as a selection unit for selecting a recommended song from a plurality of songs.

According to the present invention, there is an effect that a music piece suitable for a user's taste can be recommended without the user inputting an emotional state.

It is a figure which shows the structure of the music selection system which concerns on embodiment. It is a figure which shows the structure of the music selection apparatus. It is a figure for demonstrating the more detailed operation of the estimation part. It is a figure for demonstrating the selection of an emotion map by a matching part. It is a figure for demonstrating the information which the update part stores in the storage part. It is a flowchart which showed the process of selecting the music by the music selection apparatus.

[Overview of music selection system S]
FIG. 1 is a diagram showing a configuration of a music selection system S according to the present embodiment. The music selection system S is a system for selecting music that suits the user's taste. The music selection system S includes a plurality of communication terminals 100, a music selection device 200, and a plurality of social network servers 300. The plurality of communication terminals 100, the music selection device 200, and the plurality of social network servers 300 are connected to each other via a network N such as the Internet.

The communication terminal 100 is a device used for posting a posted message or a posted image to the social network server 300 via the network N, and is, for example, a smartphone or a tablet. Social network services are, for example, Twitter®, LiveJournal. The communication terminal 100 may be a wearable device such as Fitbit®. In this case, the communication terminal 100 measures the amount of exercise of the user and transmits the measured amount of exercise to the Fitbit WEB server, the social network server 300 linked to Fitbit, or the like.

The communication terminal 100 acquires user data associated with the user's social network account from the social network server 300 ((1) in the figure). The user data is, for example, a posted message posted by the user, a posted image, or momentum data of the user managed by the Fitbit server. The communication terminal 100 may acquire user data from each of the plurality of social network servers 300. The communication terminal 100 transmits the acquired user data to the music selection device 200 via the network N.

The music selection device 200 recommends to the user a music that suits the user's taste. More specifically, the music selection device 200 predicts the emotion of the user by analyzing the user data acquired from the communication terminal 100 ((2) in the figure). The music selection device 200 selects a music that matches the predicted user's emotion as a music that suits the user's taste. The music selection device 200 recommends the selected music to the user ((3) in the figure).

The music selection device 200 acquires feedback indicating the user's evaluation of the recommended music ((4) in the figure). For example, the music selection device 200 obtains user feedback on the recommended music by analyzing the user data posted by the user to the social network server 300 after recommending the music.

In this way, in the music selection system S, the music selection device 200 estimates the user's emotions based on the user's posted message or posted image, and recommends to the user a music that matches the estimated emotions. Therefore, the user can listen to the music he / she wants to listen to without bothering the user, and the satisfaction level of the user can be enhanced.

[Structure of music selection device 200]
FIG. 2 is a diagram showing the configuration of the music selection device 200. The music selection device 200 includes a communication unit 201, a storage unit 202, and a control unit 203. The communication unit 201 is a communication interface for communicating with the communication terminal 100 and the social network server 300 via the network N. The storage unit 202 is a storage medium such as a ROM (Read Only Memory) and a RAM (Random Access Memory).

The storage unit 202 stores a program executed by the control unit 203. In addition, the storage unit 202 stores various data used for estimating the user's emotions and recommending music to the user. Details of various data stored in the storage unit 202 will be described later.

The control unit 203 is, for example, a CPU (Central Processing Unit). The control unit 203 functions as a collection unit 1, an emotion estimation unit 2, and a recommendation unit 3 by executing the program stored in the storage unit 202. The collecting unit 1 acquires the user data via the communication unit 201, and classifies the posted message in the user data into positive, negative, and neutral emotions. The collecting unit 1 has a data acquisition unit 11 and a feature extraction unit 12.

The data acquisition unit 11 acquires the user data associated with the user's account on the social network server 300 from the communication terminal 100 via the communication unit 201. The account is, for example, a user ID of a social network. The data acquisition unit 11 acquires, for example, user data for the last three years as past user data. In addition, the data acquisition unit 11 periodically acquires user data added thereafter from the communication terminal 100 or the social network server 300. For example, the data acquisition unit 11 acquires the user data added thereafter every day.

The feature extraction unit 12 extracts a feature indicating emotion from the posted message in the user data acquired by the data acquisition unit 11. The feature extraction unit 12 shows the linguistic features shown in the posted message or the like posted by the user of the communication terminal 100 on the social network server 300, and the posted image such as a photograph posted by the user of the communication terminal 100 on the social network server 300. The non-verbal features to be extracted from the user data. The feature extraction unit 12 extracts online features, social features, and physical features that are considered to have a correlation with the user's emotions from the posted message and the posted image. Hereinafter, various features extracted by the feature extraction unit 12 will be described.

(Features on the net)
The feature extraction unit 12 extracts the number of search tags for searching the posted message in the user data. The search tag is, for example, a hashtag when the social network service is Twitter. In addition, the feature extraction unit 12 extracts the number of posted messages per day or the number of posted images designated as favorites by other users. When the social network service is Twitter, the feature extraction unit 12 extracts the number of tweets generated by an application other than Twitter.

The feature extraction unit 12 extracts the day of the week on which the user posted the posted message or posted image. For example, the feature extraction unit 12 extracts whether the day of the week on which the user posted the posted message or posted image is a weekday from Monday to Friday, or a Saturday or Sunday holiday.

The feature extraction unit 12 counts the number of positive actions and the number of passive actions in a predetermined period. The predetermined period is a period for statistically estimating the emotion of the user, for example, 3 days. A positive action is to post a posted message. Passive action is to repost a post message that you or another user has posted in the past. The feature extraction unit 12 extracts the ratio of the number of positive actions to the total of the number of positive actions and the number of passive actions.

(Social features)
Users with a large number of posted messages tend to be more emotionally positive. Therefore, the feature extraction unit 12 extracts the number of posted messages per day. Also, the more messages posted (eg, mentions on Twitter) that refer to other users, the more positive the emotions tend to be. Therefore, the feature extraction unit 12 extracts the number of mentions of other users in the posted message posted in a predetermined period in the user data. The predetermined period is a period for statistically estimating the user's emotions. In this case, the number of references to other users in the posted message may be the number of references to one other user, or the total number of references to a plurality of other users.

Also, when a user refers to a particular user, the positiveness of the user's emotions tends to be high. Therefore, the feature extraction unit 12 predicts the user's emotion when the user refers to a specific other user in the posted message by the feature extracted from the same posted message, and the predicted emotion is referred to by another user. It is stored in the storage unit 202 in association with the user. The feature extraction unit 12 has another feature referred to as a feature associated with the emotion stored in the storage unit 202 when the data acquisition unit 11 acquires a posted message re-referenced by another memorized user. Extract users.

For example, the feature extraction unit 12 predicts that the user's emotion is "positive" from the character string "good" when the posted message "I'm glad I met Mr. XX" is extracted. At this time, the feature extraction unit 12 stores the other user “○○” in association with the emotion “positive”. When the data acquisition unit 11 acquires the posted message "I met Mr. XX" from the next time onward, the feature extraction unit 12 extracts another user "XX" as a feature associated with the emotion "positive". ..

(Physical characteristics)
When the position information is associated with the posted message in the user data acquired by the data acquisition unit 11, the feature extraction unit 12 associates the position indicated by the position information with the emotion indicated in the same posted message and stores the storage unit 202. To memorize. The position information is, for example, GPS (Global Positioning System) information.

For example, the feature extraction unit 12 determines that the user has a positive emotion from the character string "refresh" when the message "the air is clean and refreshed" is associated with the location information indicating the △△ city. Predict and store the location information indicating the △△ city in the storage unit 202 in association with positive emotions. When the data acquisition unit 11 acquires the posted message or the posted image including the position information indicating the △△ city from the next time onward, the feature extraction unit 12 refers to the storage unit 202 as a feature associated with a positive emotion. This position information is extracted.

Also, users tend to be more positive when they share posts on one social network service with other social network services. Therefore, the feature extraction unit 12 extracts the number of posts that disclose the amount of exercise that the user has exercised in the social network service. For example, the feature extraction unit 12 extracts the number of user exercises measured by Fitbit posted on Twitter when the social network is Twitter.

In addition, after the user exercises, the degree of positiveness tends to be higher than before the exercise. Therefore, the feature extraction unit 12 extracts the number of steps from the user data when the user data indicating the number of steps taken by the user is acquired.

[Estimation of user's emotion]
The emotion estimation unit 2 estimates the user's emotion based on the user data acquired by the data acquisition unit 11. The emotion estimation unit 2 has a data learning unit 21 and a prediction model unit 22. The data learning unit 21 learns what kind of emotion of the user the various features extracted by the feature extraction unit 12 correspond to. The data learning unit 21 calculates the correlation degree between the result of estimating the user's emotion based on the posted message expressing the user's emotion and the various features extracted by the feature extraction unit 12, thereby making the posted message. Make it possible to estimate the user's emotions without using it.

First, the data learning unit 21 estimates the user's emotion in the user data by calculating the ratio of emotions (value between 0 and 1) appearing in the posted message within a predetermined period. The data learning unit 21 estimates the user's emotions by, for example, the following formula.

(Number 1)
M = N _pos / (N _pos + N _neg + N _neu ) ... (1)

In the formula, M is an emotion and N _pos is the number of posted messages indicating a positive emotion. N _neg is the number of posted messages that indicate negative emotions. N _neu is the number of neutral posted messages. A neutral post message is a post message that does not show any positive or negative emotions, or a post message that shows the same degree of positive and negative emotions.

When the data learning unit 21 acquires the posted message including the character string "Today was a wonderful day" in the data acquisition unit 11, the data learning unit 21 recognizes the character string "excellent" as a positive term and acquires it. Judge that the posted message is a posted message showing positive emotions.

On the other hand, the data learning unit 21 considers the character string "unfortunate" to be a negative term when the data acquisition unit 11 acquires a posted message including the character string "I had something unfortunate today". Recognize and determine that the acquired posted message is a posted message showing negative emotions. When the data learning unit 21 does not recognize the character string indicating the user's emotion, or when it recognizes the character string indicating the positive and negative emotions to the same extent, the data learning unit 21 makes this posted message a neutral posted message. Is determined to be. Further, the data learning unit 21 estimates the excitement degree in the user's emotion by the same method as the positive degree in the user's emotion.

Subsequently, the data learning unit 21 obtains the degree of correlation between the estimated emotion and the feature extracted by the feature extraction unit 12. For example, the data learning unit 21 obtains a Pearson coefficient indicating the degree of correlation between the estimated emotion and the feature extracted by the feature extraction unit 12. The data learning unit 21 learns that the emotion whose degree of correlation with the feature extracted by the feature extraction unit 12 is equal to or higher than the threshold value is the emotion appearing in the feature extracted by the feature extraction unit 12. The data learning unit 21 notifies the prediction model unit 22 of the learned result.

When the prediction model unit 22 receives the input of the feature extracted by the feature extraction unit 12, the prediction model unit 22 outputs information indicating an emotion corresponding to the input feature. Specifically, the prediction model unit 22 has a memory for storing data associated with the features extracted by the feature extraction unit 12 and the user's emotions, and inputs the features extracted by the feature extraction unit 12. When received, it outputs information indicating the emotion associated with the input feature.

[Recommendation of songs suitable for user's emotions]
The recommendation unit 3 recommends to the user a music that suits the user's taste. The recommendation unit 3 has a map management unit 31, a matching unit 32, and a selection unit 33.

The map management unit 31 generates an emotion map showing the relationship between a plurality of songs and human emotions. The map management unit 31 generates, for example, an emotion map for each user, and stores the generated emotion map in the storage unit 202.

The detailed operation of the map management unit 31 will be described with reference to FIG. As a preprocessing before generating the emotion map, the map management unit 31 causes the prediction model unit 22 to output information indicating the user's emotion when listening to the music, and outputs the user's emotion output by the prediction model unit 22. The indicated information is stored in the storage unit 202 in association with the music that the user is listening to.

The map management unit 31 classifies the music into one of the three types of emotions (positive, negative, and neutral) based on the acquired information indicating the user's emotions ((1) in FIG. 3). )). For example, when the prediction model unit 22 outputs information indicating that the user's emotions when listening to a certain music are positive, the map management unit 31 classifies the music into a positive type. The map management unit 31 arranges each musical piece in a two-dimensional space (hereinafter, referred to as an emotion two-dimensional space) indicating the type of emotion ((2) in FIG. 3). The emotion two-dimensional space is, for example, an induced excitement (valence-arousal, VA) emotion space that expresses the type of emotion in the two-dimensional space, and the map management unit 31 arranges each song in this emotion two-dimensional space. By doing so, it is possible to relate a plurality of songs to human emotions.

The emotion of a piece of music is represented by, for example, high-dimensional data to which a vector of 43 values is assigned to each piece of music. The t-SNE algorithm is a tool for visualizing high-dimensional data, and converts high-dimensional data indicating the emotion of a music into two-dimensional data corresponding to the emotion two-dimensional space. The map management unit 31 uses the t-SNE algorithm to convert similarities between data points into simultaneous probabilities, and Kullback-Leibler information between the simultaneous probabilities of low-dimensional data and the simultaneous probabilities of high-dimensional data to be placed. Try to minimize the amount. The map management unit 31 stores in the storage unit 202 three types of emotion maps indicating the positions of the music in the user's emotion two-dimensional space generated in this way ((3) in FIG. 3).

The matching unit 32 selects an emotion map that matches the user's emotion estimated by the emotion estimation unit 2 from among the emotion maps for the three emotions of positive, negative, and neutral. When recommending a recommended song to the user, the matching unit 32 selects an emotion map that matches the user's emotion from the user's emotion map.

The selection of the emotion map by the matching unit 32 will be described with reference to FIG. FIG. 4 shows an emotion map created by the prediction model unit 22. FIG. 4 (a) shows an emotion map corresponding to positive emotions, FIG. 4 (b) shows an emotion map corresponding to neutral emotions, and FIG. 4 (c) corresponds to negative emotions. Show the emotion map.

As shown in FIG. 4A, the map management unit 31 arranges a plurality of songs in the two-dimensional space in the emotion map. The circles placed on the emotion map indicate each song. The closer the distance on the emotion map between songs, the more similar the characteristics of the songs.

In the emotion map of FIG. 4A, music suitable for a state with a high degree of positiveness is arranged. When the positiveness of the user's emotion output by the prediction model unit 22 is higher than the first threshold value, the matching unit 32 selects an emotion map corresponding to the positive emotion. The first threshold is set so that, for example, the number of songs arranged on the three emotion maps is equal to each other.

FIG. 4B shows an emotional map corresponding to a neutral emotion. In this emotion map, songs suitable for a state in which the degree of positiveness is lower than that of the emotion map of FIG. 4A are arranged. The matching unit 32 selects an emotion map corresponding to a neutral emotion when the positiveness of the user's emotion output by the prediction model unit 22 is equal to or less than the first threshold value and higher than the second threshold value. The second threshold is set so that the number of songs arranged on the three emotion maps is equal to each other, as in the case of the first threshold, for example.

FIG. 4 (c) shows an emotion map corresponding to negative emotions. In this emotion map, songs having a lower positive degree than the emotion map of FIG. 4B are arranged. The matching unit 32 selects an emotion map corresponding to a negative emotion when the positive degree of the user's emotion estimated by the data learning unit 21 is equal to or less than the second threshold value.

[Selection of recommended songs]
Returning to the description of FIG. The selection unit 33 acquires music data indicating the characteristics of the plurality of songs from the storage unit 202. The music data is, for example, data indicating the position of the music in the emotion map stored in the storage unit 202. The selection unit 33 selects a recommended music suitable for the user's emotions estimated by the prediction model unit 22 based on the relationship between the music stored in the storage unit 202 and the emotions.

More specifically, the selection unit 33 is recommended for recommending to the user from a plurality of songs by comparing the characteristics of the plurality of songs indicated by the acquired music data with the emotions of the user estimated by the prediction model unit 22. Select a song. In the example of this embodiment, first, the selector 33 selects the highest music with popularity Q ₀ as the first recommended music in emotion map matching unit 32 has selected. Popularity _{Q 0} is shown by the following equation (2).

(Number 2)
Q ₀ = number of times you listened to music ÷ total number of times you listened to all music ... (2)

The selection unit 33 selects a recommended song in the emotion map selected by the matching unit 32 from among the plurality of emotion maps. By adopting such a configuration, the selection unit 33 reduces the processing load of the processor for selecting the recommended music as compared with the case of selecting the recommended music from the emotion map in which all the songs are arranged. Can be done.

Further, in the selection of the recommended music from the second time onward, the selection unit 33 estimates the user's preference based on the feedback when the user listens to the recommended music selected in the past, and newly based on the estimated preference. Select a recommended song. More specifically, the selection unit 33 selects the recommended music from the second time onward in consideration of the implicit feedback on the recommended music of the user. At this time, the selection unit 33 estimates the user's preference based on the implicit feedback without requiring the user's explicit instruction or evaluation. Implicit feedback includes a click operation to select a song, listening time (lt), song repeat, adding a song to a favorite / playlist (fv), listening count (nl), and the like. The present inventor has found that the implicit feedback can be quantified based on the user's preference for the music and the position of the music on the emotion map, as shown in the following equation (3).

(Number 3)
R _t (u, s, m) = R _{pref, t} (u, s, m) + R _{d, t} (u, s, m)… (3)

In the formula, R _{pref, t} indicates the user's preference for the music, and R _{d, t} indicate the reward given based on the position of the music on the emotion map. R _{pref, t} and R _{d, t} are represented by the following equations (4), respectively.

In the equation, _kk is the weight of each factor of suggestive feedback, and rk _{, t} (u, s, m) is the value of factor k for music s and K = {lt, fv, nl}. is there.

The selection unit 33 selects a new recommended song based on the degree of similarity with the recommended song selected immediately before. The degree of similarity is the distance between two songs in the emotion map, and the smaller the distance between the two songs, the higher the degree of similarity. The selection unit 33 selects recommended songs that are different _{between the search period T 1} and the utilization period T _2. The search period T ₁ is a period for searching the user's preference, and the utilization period T ₂ is a period after the identification of the user's preference is completed.

Selection unit 33, the search period T ₁ searches for new songs by selecting music of different trends and recent recommended music, to learn about the user's preference. Selector 33, during the search period T ₁ which is searching the preference of the user, the similarity of the recommendations music that was selected directly before is to select a smaller recommended music than the predetermined threshold value. The predetermined threshold value may be specified by the user, for example, so that the distance in the emotion map from the recommended song selected immediately before becomes larger than _{davg, which is the average distance between all the songs in the emotion map.} The value. d _avg, for example, music _s 1 on emotion map corresponding to positive _feelings, s 2, · · _{·, if s 100} is arranged, the music _{s j -s k (j = 1,2} , · ..., 100, k = 1, 2, ..., 100, j ≠ k) is the average value of the distances in all combinations.

In the search period T ₁ , when the distance in the emotion map between the new recommended song and the immediately selected recommended song is _{larger than davg} , d (st) is set to R _{d, t} (u, s, m). , St-1) are added. On the other hand, in the search period T ₁ , when the distance in the emotion map between the new recommended song and the immediately selected recommended song is _{smaller than davg , R d, t} (u, s, m) to d. (St, st-1) is subtracted. By doing so, the selection unit 33 can select various types of music, so that it is possible to specify what kind of music the user likes in a short period of time.

The selection unit 33, the similarity of the recommended music selected just before the utilization period T ₂ of the after the search preferences of the user has finished selecting a recommended song more than a predetermined threshold value. Predetermined threshold, for example, may be specified by the user, the distance in the emotional map of the recommended music that was selected directly before is a value of about smaller than d _avg. More specifically, the selection unit 33, in the utilization period T _2, when the distance in the emotional map between the recommended music selected a new recommended music before and is smaller than d _avg is R _{d, t} ( d (st, st-1) is added to u, s, m). On the other hand, in the utilization period T _2, when the distance in the emotional map between the recommended music selected a new recommended music before and is greater than d _{avg is, R d, t (u,} s, m) from the d (St, st-1) is subtracted. By doing so, the selection unit 33 can easily select the music that the user likes.

The selection unit 33 calculates the probability P (u, s, m) of the music in the _{search period T 1} and the utilization period T _2. More specifically, the selection unit 33 calculates the probability P (u, s, m) of the music by the following equation (5) using the softmax method. The selection unit 33 selects the music having the highest probability as the recommended music.

In the equation, the update function _Qt (u, s, m) is defined by the following equation (6).
(Number 6)
Q _t (u, s, m) = Q _t-1 (u, s, m) + α _t (R _t (u, s, m)-Q _t-1 (u, s, m))… (6 )

In the equation (6), α _t is a coefficient indicating how much priority is given to the newly acquired information over the old information. When α _t is 0, it indicates that the selection unit 33 does not consider the newly acquired information at all and considers only the old information. When α _t is 1, it indicates that the selection unit 33 considers only the newly acquired information and does not consider the old information.

The selection unit 33 recommends the selected recommended music to the user. For example, the selection unit 33 transmits a message recommending the selected recommended music to the user's communication terminal 100 via the communication unit 201. Further, the selection unit 33 reads the voice data of the selected recommended music from the storage unit 202, and transmits the voice data of the recommended music to the user's communication terminal 100 together with the message recommending the selected recommended music.

[Detection of changes in palatability]
Even for the same user, the song you want to listen to may change depending on your emotions when listening to the song. Therefore, the map management unit 31 functions as an update unit that changes the relationship (emotion map) stored in the storage unit 202 based on the feedback when the user listens to the recommended music selected by the selection unit 33 in the past. .. By selecting the recommended music using the updated emotion map, the selection unit 33 can recommend the appropriate recommended music to the user even when the user's preference changes due to the change in the user's emotion. it can.

Map management unit 31, the utilization period T ₂ the search for the preference of the user by the selection unit 33 has been completed, based on the feedback of when the selection unit 33 heard the user the recommended music that was selected in the past, the preference of the user It functions as a detector that monitors sexual changes. This feedback is, for example, implicit feedback. The map management unit 31 detects a change in the user's preference by analyzing the feedback when the user listens to the recommended music selected by the selection unit 33 in the past. For example, the map management unit 31 detects a change in the user's preference when the change in the reward shown in the equation (3) is specified by the Page-Hinkley Test (PHT) (about the Page-Hinkley test). Is Hartland, C., Gelly, S., Baskiotis, N., Teytaud, O., & Sebag, M. (2006). Multi-armed band it, dynamic environment sand meta-bandits. Preprint. Http: // hal See .archives-ouvertes.fr/hal-00113668/en/.).

The map management unit 31 changes the relationship between the plurality of songs stored in the storage unit 202 and human emotions based on the feedback when the user hears the recommended music selected by the selection unit 33 in the past. For example, when the map management unit 31 detects a change in the user's preference by analyzing the user's implicit feedback, the map management unit 31 updates the emotion map showing the relationship between the plurality of songs and the human emotion.

The map management unit 31 requantifies the emotions of each song by referring to the implicit feedback on the user's recommended song. The map management unit 31 uses the same t-SNE algorithm as the prediction model unit 22, but reflects the implicit feedback shown in the equation (6) in the emotion map. The map management unit 31 determines this in the emotion map when, for example, the user's implicit feedback after recommending the recommended song corresponding to the negative emotion to the user indicates that the user's evaluation of the recommended song is not good. Move the position of the song.

The selection unit 33 refers to a new emotion map updated based on the user's preference estimated by the map management unit 31 based on the feedback when the user listens to the recommended music selected in the past. Select a recommended song. By adopting such a configuration, the selection unit 33 can improve the method of selecting the recommended music so as to suit the user's preference when the recommended music selected in the past does not suit the user's preference. it can.

Selection unit 33, if the user's preference changes in utilization period T _2, it is desirable to explore music that fits the user's preference again. Therefore, when the map management unit 31 detects that the user's preference has changed in the _{utilization period T 2} , the selection unit 33 returns the _{utilization period T 2} to the search period T _{1. By returning the utilization period T 2} to the search period T ₁ , the selection unit 33 gives priority to selecting a song having a tendency different from the most recently recommended song, and restarts the search for the user's preference.

[Data stored in the storage unit 202]
FIG. 5 is a diagram for explaining the data stored in the storage unit 202. In the example of the present embodiment, the music selection device 200 includes a cloud-based server, and the storage unit 202 includes a first storage unit A1, a second storage unit A2, and a third storage unit A3.

The first storage unit A1 is, for example, a data set acquired from the Live Journal2 database, which is a social blog service. The first storage unit A1 stores the history of songs that the user has heard in different emotions. The first storage unit A1 stores the blog article, the emotion when listening to the music, and the music heard in association with each other. For example, in the first storage unit A1, the emotion 1 when listening to the music 1 and the article 1 are stored in association with the music 1. The map management unit 31 generates an emotion map by referring to the emotions associated with the music stored in the first storage unit.

Since the emotion map generated by the map management unit 31 is different for each user, the second storage unit A2 stores the emotion map in association with the user ID. Further, in the first storage unit A1, the user's emotions estimated by the data learning unit 21 and the voice data of a plurality of musical pieces are stored in association with each other.

The prediction model unit 22 stores the generated emotion map in the second storage unit A2. The emotion map stored in the second storage unit A2 is updated by the map management unit 31.

In the example of FIG. 5, the emotion map 1 is associated with the user ID 1, and the emotion map 2 is associated with the user ID 2. In the second storage unit A2, emotion maps corresponding to positive, negative, and neutral emotions are associated with one user ID, respectively. The matching unit 32 selects an emotion map for selecting a recommended song by the selection unit 33 from the emotion maps corresponding to positive, negative, and neutral emotions, and acquires the selected emotion map from the second storage unit A2. ..

In the third storage unit A3, social network accounts of a plurality of users are stored in association with the user ID. In the example of FIG. 5, as a social network account, Twitter ID1 and Instagram (registered trademark) ID1 are associated with user ID1. The map management unit 31 acquires the account of the social network service used by the user from the communication terminal 100 via the communication unit 201 and stores it in the third storage unit A3. The data acquisition unit 11 acquires a user account such as a Twitter ID from the third storage unit A3, and periodically obtains user data associated with the acquired account on the social network server 300 from the communication terminal 100 or the social network server 300. get.

[Music selection process by the music selection device 200]
FIG. 6 is a flowchart showing a process of selecting a musical piece by the music selection device 200. This processing procedure starts, for example, when the map management unit 31 receives a request from a new user to recommend a song.

First, the map management unit 31 acquires an account of the social network service used by the user from the communication terminal 100 (S101). The data acquisition unit 11 acquires the user data associated with the user's social network service account on the social network server 300 from the communication terminal 100 (S102). The feature extraction unit 12 extracts features from the user data acquired by the data acquisition unit 11 (S103).

The data learning unit 21 estimates the user's emotion by analyzing the message posted by the user, and calculates the degree of correlation between the estimated user's emotion and the feature extracted by the feature extraction unit 12, thereby calculating a prediction model. Part 22 is updated (S104). After that, when a feature showing a significant degree of correlation with the user's emotion is input, the prediction model unit 22 outputs data showing the emotion corresponding to the input feature (S105). After that, the selection unit 33 selects the music corresponding to the emotion input from the prediction model unit 22 by referring to the emotion map, and recommends it to the communication terminal 100 (S106).

[Action and effect according to the present invention]
According to the present embodiment, the selection unit 33 recommends from a plurality of songs by comparing the user's emotions estimated by the emotion estimation unit 2 with the characteristics of a plurality of songs in the music data indicating the characteristics of the songs. Select a song. The emotion estimation unit 2 estimates the user's emotion based on the characteristics of the behavior that the user posts on the social network. Therefore, the music selection device 200 can recommend a music that suits the user's taste without the user inputting an emotional state.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes can be made within the scope of the gist thereof. is there. For example, the specific embodiment of the distribution / integration of the device is not limited to the above embodiment, and all or a part thereof may be functionally or physically distributed / integrated in any unit. Can be done. Also included in the embodiments of the present invention are new embodiments resulting from any combination of the plurality of embodiments. The effect of the new embodiment produced by the combination has the effect of the original embodiment together.

1 Collection unit 2 Emotion estimation unit 3 Recommendation unit 11 Data acquisition unit 12 Feature extraction unit 21 Data learning unit 22 Prediction model unit 31 Map management unit 32 Matching unit 33 Selection unit 100 Communication terminal 200 Music selection device 201 Communication unit 202 Storage unit 203 Control unit 300 Social network server A1 1st storage unit A2 2nd storage unit A3 3rd storage unit

Claims (9)

A data acquisition unit that acquires user data associated with the user's social network account,
An emotion estimation unit that estimates the user's emotions based on the user data acquired by the data acquisition unit, and an emotion estimation unit.
By acquiring music data indicating the characteristics of a plurality of songs and comparing the emotions of the user estimated by the emotion estimation unit with the characteristics of the plurality of songs indicated by the music data, recommended songs from the plurality of songs. After selecting the recommended music and searching for the preference of the user, the recommended music having a similarity to the recommended music selected immediately before is smaller than a predetermined threshold is selected, and the recommended music is selected. After the search for the user's preference is completed, a selection unit that selects the recommended music whose similarity to the recommended music selected immediately before is equal to or higher than a predetermined threshold, and
Music selection device with. Further having a storage unit for storing the relationship between the plurality of songs and human emotions,
The selection unit selects the recommended music suitable for the user's emotions estimated by the emotion estimation unit based on the relationship stored in the storage unit.
The music selection device according to claim 1. The storage unit stores the relationship by storing the positions of the plurality of songs in the emotion map showing the characteristics of the songs.
The music selection device according to claim 2. The storage unit stores the relationship between the plurality of songs and the user's emotions for each type of the user's emotions.
The music selection device according to claim 2 or 3. The selection unit further includes an update unit that changes the relationship stored in the storage unit based on feedback when the user listens to the recommended music selected in the past.
The music selection device according to any one of claims 2 to 4. The selection unit selects a new recommended song based on the user's preference estimated based on the feedback when the user listens to the recommended song selected in the past.
The music selection device according to any one of claims 1 to 5. After the search for the user's preference is completed, a detection unit that monitors the change in the user's preference based on the feedback when the user listens to the recommended song selected in the past is further provided.
When the detection unit detects that the preference has changed, the selection unit resumes the search for the user's preference.
The music selection device according to any one of claims 1 to 6. Computer runs,
A step to acquire user data associated with a user's social network account and music data indicating the characteristics of a plurality of songs, and
A step of estimating the emotion of the user based on the acquired user data, and
A step of selecting a recommended song from a plurality of songs by comparing the estimated emotion of the user with the characteristics of the plurality of songs indicated by the song data .
After selecting the recommended song, while searching for the preference of the user, a step of selecting the recommended song whose similarity to the recommended song selected immediately before is smaller than a predetermined threshold value, and
After the search for the user's preference is completed, a step of selecting the recommended music whose similarity to the recommended music selected immediately before is equal to or higher than a predetermined threshold value , and
Music selection method having. Computer,
Data acquisition department, which acquires user data associated with the user's social network account,
Based on the user data acquired by the data acquisition unit, an emotion estimation unit that estimates the user's emotions, and music data indicating the characteristics of a plurality of songs are acquired, and the user's emotions estimated by the emotion estimation unit. By comparing the characteristics of the plurality of songs indicated by the music data with, a recommended music is selected from the plurality of songs, and after the recommended music is selected, while the user's preference is searched for. After selecting the recommended song whose similarity with the recommended song selected immediately before is smaller than a predetermined threshold and completing the search for the preference of the user, the similarity with the recommended song selected immediately before is determined. Selection unit for selecting the recommended music above the threshold of
A program to function as.

Images