个人网站导航html源码,江阴设计公司,宿迁网络推广公司,奢做品二手 哪个网站好请安装TensorFlow1.0#xff0c;Python3.5 项目地址#xff1a;GitHub - xiaobingchan/movie_recommender: MovieLens based recommender system.使用MovieLens数据集训练的电影推荐系统。
前言 本项目使用文本卷积神经网络#xff0c;并使用MovieLens数据集完成电影推荐的…请安装TensorFlow1.0Python3.5 项目地址GitHub - xiaobingchan/movie_recommender: MovieLens based recommender system.使用MovieLens数据集训练的电影推荐系统。
前言 本项目使用文本卷积神经网络并使用MovieLens数据集完成电影推荐的任务。 推荐系统在日常的网络应用中无处不在比如网上购物、网上买书、新闻app、社交网络、音乐网站、电影网站等等等等有人的地方就有推荐。根据个人的喜好相同喜好人群的习惯等信息进行个性化的内容推荐。比如打开新闻类的app因为有了个性化的内容每个人看到的新闻首页都是不一样的。 这当然是很有用的在信息爆炸的今天获取信息的途径和方式多种多样人们花费时间最多的不再是去哪获取信息而是要在众多的信息中寻找自己感兴趣的这就是信息超载问题。为了解决这个问题推荐系统应运而生。 协同过滤是推荐系统应用较广泛的技术该方法搜集用户的历史记录、个人喜好等信息计算与其他用户的相似度利用相似用户的评价来预测目标用户对特定项目的喜好程度。优点是会给用户推荐未浏览过的项目缺点呢对于新用户来说没有任何与商品的交互记录和个人喜好等信息存在冷启动问题导致模型无法找到相似的用户或商品。 为了解决冷启动的问题通常的做法是对于刚注册的用户要求用户先选择自己感兴趣的话题、群组、商品、性格、喜欢的音乐类型等信息比如豆瓣FM 先来看看数据 本项目使用的是MovieLens 1M 数据集包含6000个用户在近4000部电影上的1亿条评论。 数据集分为三个文件用户数据users.dat电影数据movies.dat和评分数据ratings.dat。
用户数据 分别有用户ID、性别、年龄、职业ID和邮编等字段。
数据中的格式UserID::Gender::Age::Occupation::Zip-code
Gender is denoted by a “M” for male and “F” for female Age is chosen from the following ranges:
1: “Under 18” 18: “18-24” 25: “25-34” 35: “35-44” 45: “45-49” 50: “50-55” 56: “56” Occupation is chosen from the following choices:
0: “other” or not specified 1: “academic/educator” 2: “artist” 3: “clerical/admin” 4: “college/grad student” 5: “customer service” 6: “doctor/health care” 7: “executive/managerial” 8: “farmer” 9: “homemaker” 10: “K-12 student” 11: “lawyer” 12: “programmer” 13: “retired” 14: “sales/marketing” 15: “scientist” 16: “self-employed” 17: “technician/engineer” 18: “tradesman/craftsman” 19: “unemployed” 20: “writer”
其中UserID、Gender、Age和Occupation都是类别字段其中邮编字段是我们不使用的。 电影数据 分别有电影ID、电影名和电影风格等字段。
数据中的格式MovieID::Title::Genres
Titles are identical to titles provided by the IMDB (including year of release) Genres are pipe-separated and are selected from the following genres:
Action Adventure Animation Children’s Comedy Crime Documentary Drama Fantasy Film-Noir Horror Musical Mystery Romance Sci-Fi Thriller War Western
MovieID是类别字段Title是文本Genres也是类别字段 评分数据 分别有用户ID、电影ID、评分和时间戳等字段。
数据中的格式UserID::MovieID::Rating::Timestamp
UserIDs range between 1 and 6040 MovieIDs range between 1 and 3952 Ratings are made on a 5-star scale (whole-star ratings only) Timestamp is represented in seconds since the epoch as returned by time(2) Each user has at least 20 ratings
评分字段Rating就是我们要学习的targets时间戳字段我们不使用。 说说数据预处理 UserID、Occupation和MovieID不用变。 Gender字段需要将‘F’和‘M’转换成0和1。 Age字段要转成7个连续数字0~6。 Genres字段是分类字段要转成数字。首先将Genres中的类别转成字符串到数字的字典然后再将每个电影的Genres字段转成数字列表因为有些电影是多个Genres的组合。 Title字段处理方式跟Genres字段一样首先创建文本到数字的字典然后将Title中的描述转成数字的列表。另外Title中的年份也需要去掉。 Genres和Title字段需要将长度统一这样在神经网络中方便处理。空白部分用‘ PAD ’对应的数字填充。 数据预处理的代码可以在项目中找到load_data函数
模型设计
通过研究数据集中的字段类型我们发现有一些是类别字段通常的处理是将这些字段转成one hot编码但是像UserID、MovieID这样的字段就会变成非常的稀疏输入的维度急剧膨胀这是我们不愿意见到的毕竟我这小笔记本不像大厂动辄能处理数以亿计维度的输入 所以在预处理数据时将这些字段转成了数字我们用这个数字当做嵌入矩阵的索引在网络的第一层使用了嵌入层维度是N32和N16。 电影类型的处理要多一步有时一个电影有多个电影类型这样从嵌入矩阵索引出来是一个n32的矩阵因为有多个类型嘛我们要将这个矩阵求和变成132的向量。 电影名的处理比较特殊没有使用循环神经网络而是用了文本卷积网络下文会进行说明。 从嵌入层索引出特征以后将各特征传入全连接层将输出再次传入全连接层最终分别得到1200的用户特征和电影特征两个特征向量。 我们的目的就是要训练出用户特征和电影特征在实现推荐功能时使用。得到这两个特征以后就可以选择任意的方式来拟合评分了。我使用了两种方式一个是上图中画出的将两个特征做向量乘法将结果与真实评分做回归采用MSE优化损失。因为本质上这是一个回归问题另一种方式是将两个特征作为输入再次传入全连接层输出一个值将输出值回归到真实评分采用MSE优化损失。 实际上第二个方式的MSE loss在0.8附近第一个方式在1附近5次迭代的结果。
文本卷积网络 网络看起来像下面这样
图片来自Kim Yoon的论文Convolutional Neural Networks for Sentence Classification
将卷积神经网络用于文本的文章建议你阅读Understanding Convolutional Neural Networks for NLP 网络的第一层是词嵌入层由每一个单词的嵌入向量组成的嵌入矩阵。下一层使用多个不同尺寸窗口大小的卷积核在嵌入矩阵上做卷积窗口大小指的是每次卷积覆盖几个单词。这里跟对图像做卷积不太一样图像的卷积通常用2x2、3x3、5x5之类的尺寸而文本卷积要覆盖整个单词的嵌入向量所以尺寸是单词数向量维度比如每次滑动3个4个或者5个单词。第三层网络是max pooling得到一个长向量最后使用dropout做正则化最终得到了电影Title的特征。
核心代码讲解 完整代码请见项目
#嵌入矩阵的维度 embed_dim 32 #用户ID个数 uid_max max(features.take(0,1)) 1 # 6040 #性别个数 gender_max max(features.take(2,1)) 1 # 1 1 2 #年龄类别个数 age_max max(features.take(3,1)) 1 # 6 1 7 #职业个数 job_max max(features.take(4,1)) 1# 20 1 21
#电影ID个数 movie_id_max max(features.take(1,1)) 1 # 3952 #电影类型个数 movie_categories_max max(genres2int.values()) 1 # 18 1 19 #电影名单词个数 movie_title_max len(title_set) # 5216
#对电影类型嵌入向量做加和操作的标志考虑过使用mean做平均但是没实现mean combiner sum
#电影名长度 sentences_size title_count # 15 #文本卷积滑动窗口分别滑动2, 3, 4, 5个单词 window_sizes {2, 3, 4, 5} #文本卷积核数量 filter_num 8
#电影ID转下标的字典数据集中电影ID跟下标不一致比如第5行的数据电影ID不一定是5 movieid2idx {val[0]:i for i, val in enumerate(movies.values)} 超参 # Number of Epochs num_epochs 5 # Batch Size batch_size 256
dropout_keep 0.5 # Learning Rate learning_rate 0.0001 # Show stats for every n number of batches show_every_n_batches 20
save_dir ./save 输入 定义输入的占位符
def get_inputs(): uid tf.placeholder(tf.int32, [None, 1], nameuid) user_gender tf.placeholder(tf.int32, [None, 1], nameuser_gender) user_age tf.placeholder(tf.int32, [None, 1], nameuser_age) user_job tf.placeholder(tf.int32, [None, 1], nameuser_job) movie_id tf.placeholder(tf.int32, [None, 1], namemovie_id) movie_categories tf.placeholder(tf.int32, [None, 18], namemovie_categories) movie_titles tf.placeholder(tf.int32, [None, 15], namemovie_titles) targets tf.placeholder(tf.int32, [None, 1], nametargets) LearningRate tf.placeholder(tf.float32, name LearningRate) dropout_keep_prob tf.placeholder(tf.float32, name dropout_keep_prob) return uid, user_gender, user_age, user_job, movie_id, movie_categories, movie_titles, targets, LearningRate, dropout_keep_prob 构建神经网络 定义User的嵌入矩阵 def get_user_embedding(uid, user_gender, user_age, user_job): with tf.name_scope(user_embedding): uid_embed_matrix tf.Variable(tf.random_uniform([uid_max, embed_dim], -1, 1), name uid_embed_matrix) uid_embed_layer tf.nn.embedding_lookup(uid_embed_matrix, uid, name uid_embed_layer) gender_embed_matrix tf.Variable(tf.random_uniform([gender_max, embed_dim // 2], -1, 1), name gender_embed_matrix) gender_embed_layer tf.nn.embedding_lookup(gender_embed_matrix, user_gender, name gender_embed_layer) age_embed_matrix tf.Variable(tf.random_uniform([age_max, embed_dim // 2], -1, 1), nameage_embed_matrix) age_embed_layer tf.nn.embedding_lookup(age_embed_matrix, user_age, nameage_embed_layer) job_embed_matrix tf.Variable(tf.random_uniform([job_max, embed_dim // 2], -1, 1), name job_embed_matrix) job_embed_layer tf.nn.embedding_lookup(job_embed_matrix, user_job, name job_embed_layer) return uid_embed_layer, gender_embed_layer, age_embed_layer, job_embed_layer 将User的嵌入矩阵一起全连接生成User的特征 def get_user_feature_layer(uid_embed_layer, gender_embed_layer, age_embed_layer, job_embed_layer): with tf.name_scope(user_fc): #第一层全连接 uid_fc_layer tf.layers.dense(uid_embed_layer, embed_dim, name uid_fc_layer, activationtf.nn.relu) gender_fc_layer tf.layers.dense(gender_embed_layer, embed_dim, name gender_fc_layer, activationtf.nn.relu) age_fc_layer tf.layers.dense(age_embed_layer, embed_dim, name age_fc_layer, activationtf.nn.relu) job_fc_layer tf.layers.dense(job_embed_layer, embed_dim, name job_fc_layer, activationtf.nn.relu) #第二层全连接 user_combine_layer tf.concat([uid_fc_layer, gender_fc_layer, age_fc_layer, job_fc_layer], 2) #(?, 1, 128) user_combine_layer tf.contrib.layers.fully_connected(user_combine_layer, 200, tf.tanh) #(?, 1, 200) user_combine_layer_flat tf.reshape(user_combine_layer, [-1, 200]) return user_combine_layer, user_combine_layer_flat 定义Movie ID的嵌入矩阵 def get_movie_id_embed_layer(movie_id): with tf.name_scope(movie_embedding): movie_id_embed_matrix tf.Variable(tf.random_uniform([movie_id_max, embed_dim], -1, 1), name movie_id_embed_matrix) movie_id_embed_layer tf.nn.embedding_lookup(movie_id_embed_matrix, movie_id, name movie_id_embed_layer) return movie_id_embed_layer 对电影类型的多个嵌入向量做加和 def get_movie_categories_layers(movie_categories): with tf.name_scope(movie_categories_layers): movie_categories_embed_matrix tf.Variable(tf.random_uniform([movie_categories_max, embed_dim], -1, 1), name movie_categories_embed_matrix) movie_categories_embed_layer tf.nn.embedding_lookup(movie_categories_embed_matrix, movie_categories, name movie_categories_embed_layer) if combiner sum: movie_categories_embed_layer tf.reduce_sum(movie_categories_embed_layer, axis1, keep_dimsTrue) # elif combiner mean: return movie_categories_embed_layer Movie Title的文本卷积网络实现 def get_movie_cnn_layer(movie_titles): #从嵌入矩阵中得到电影名对应的各个单词的嵌入向量 with tf.name_scope(movie_embedding): movie_title_embed_matrix tf.Variable(tf.random_uniform([movie_title_max, embed_dim], -1, 1), name movie_title_embed_matrix) movie_title_embed_layer tf.nn.embedding_lookup(movie_title_embed_matrix, movie_titles, name movie_title_embed_layer) movie_title_embed_layer_expand tf.expand_dims(movie_title_embed_layer, -1) #对文本嵌入层使用不同尺寸的卷积核做卷积和最大池化 pool_layer_lst [] for window_size in window_sizes: with tf.name_scope(movie_txt_conv_maxpool_{}.format(window_size)): filter_weights tf.Variable(tf.truncated_normal([window_size, embed_dim, 1, filter_num],stddev0.1),name filter_weights) filter_bias tf.Variable(tf.constant(0.1, shape[filter_num]), namefilter_bias) conv_layer tf.nn.conv2d(movie_title_embed_layer_expand, filter_weights, [1,1,1,1], paddingVALID, nameconv_layer) relu_layer tf.nn.relu(tf.nn.bias_add(conv_layer,filter_bias), name relu_layer) maxpool_layer tf.nn.max_pool(relu_layer, [1,sentences_size - window_size 1 ,1,1], [1,1,1,1], paddingVALID, namemaxpool_layer) pool_layer_lst.append(maxpool_layer) #Dropout层 with tf.name_scope(pool_dropout): pool_layer tf.concat(pool_layer_lst, 3, name pool_layer) max_num len(window_sizes) * filter_num pool_layer_flat tf.reshape(pool_layer , [-1, 1, max_num], name pool_layer_flat) dropout_layer tf.nn.dropout(pool_layer_flat, dropout_keep_prob, name dropout_layer) return pool_layer_flat, dropout_layer 将Movie的各个层一起做全连接
def get_movie_feature_layer(movie_id_embed_layer, movie_categories_embed_layer, dropout_layer):with tf.name_scope(movie_fc):#第一层全连接movie_id_fc_layer tf.layers.dense(movie_id_embed_layer, embed_dim, name movie_id_fc_layer, activationtf.nn.relu)movie_categories_fc_layer tf.layers.dense(movie_categories_embed_layer, embed_dim, name movie_categories_fc_layer, activationtf.nn.relu)#第二层全连接movie_combine_layer tf.concat([movie_id_fc_layer, movie_categories_fc_layer, dropout_layer], 2) #(?, 1, 96)movie_combine_layer tf.contrib.layers.fully_connected(movie_combine_layer, 200, tf.tanh) #(?, 1, 200)movie_combine_layer_flat tf.reshape(movie_combine_layer, [-1, 200])return movie_combine_layer, movie_combine_layer_flat 构建计算图
tf.reset_default_graph()
train_graph tf.Graph()
with train_graph.as_default():#获取输入占位符uid, user_gender, user_age, user_job, movie_id, movie_categories, movie_titles, targets, lr, dropout_keep_prob get_inputs()#获取User的4个嵌入向量uid_embed_layer, gender_embed_layer, age_embed_layer, job_embed_layer get_user_embedding(uid, user_gender, user_age, user_job)#得到用户特征user_combine_layer, user_combine_layer_flat get_user_feature_layer(uid_embed_layer, gender_embed_layer, age_embed_layer, job_embed_layer)#获取电影ID的嵌入向量movie_id_embed_layer get_movie_id_embed_layer(movie_id)#获取电影类型的嵌入向量movie_categories_embed_layer get_movie_categories_layers(movie_categories)#获取电影名的特征向量pool_layer_flat, dropout_layer get_movie_cnn_layer(movie_titles)#得到电影特征movie_combine_layer, movie_combine_layer_flat get_movie_feature_layer(movie_id_embed_layer, movie_categories_embed_layer, dropout_layer)#计算出评分要注意两个不同的方案inference的名字name值是不一样的后面做推荐时要根据name取得tensorwith tf.name_scope(inference):#将用户特征和电影特征作为输入经过全连接输出一个值的方案
# inference_layer tf.concat([user_combine_layer_flat, movie_combine_layer_flat], 1) #(?, 200)
# inference tf.layers.dense(inference_layer, 1,
# kernel_initializertf.truncated_normal_initializer(stddev0.01),
# kernel_regularizertf.nn.l2_loss, nameinference)#简单的将用户特征和电影特征做矩阵乘法得到一个预测评分inference tf.matmul(user_combine_layer_flat, tf.transpose(movie_combine_layer_flat)) with tf.name_scope(loss):# MSE损失将计算值回归到评分cost tf.losses.mean_squared_error(targets, inference )loss tf.reduce_mean(cost)# 优化损失
# train_op tf.train.AdamOptimizer(lr).minimize(loss) #costglobal_step tf.Variable(0, nameglobal_step, trainableFalse)optimizer tf.train.AdamOptimizer(lr)gradients optimizer.compute_gradients(loss) #costtrain_op optimizer.apply_gradients(gradients, global_stepglobal_step)
训练网络
%matplotlib inline
%config InlineBackend.figure_format retina
import matplotlib.pyplot as plt
import time
import datetimelosses {train:[], test:[]}with tf.Session(graphtrain_graph) as sess:#搜集数据给tensorBoard用# Keep track of gradient values and sparsitygrad_summaries []for g, v in gradients:if g is not None:grad_hist_summary tf.summary.histogram({}/grad/hist.format(v.name.replace(:, _)), g)sparsity_summary tf.summary.scalar({}/grad/sparsity.format(v.name.replace(:, _)), tf.nn.zero_fraction(g))grad_summaries.append(grad_hist_summary)grad_summaries.append(sparsity_summary)grad_summaries_merged tf.summary.merge(grad_summaries)# Output directory for models and summariestimestamp str(int(time.time()))out_dir os.path.abspath(os.path.join(os.path.curdir, runs, timestamp))print(Writing to {}\n.format(out_dir))# Summaries for loss and accuracyloss_summary tf.summary.scalar(loss, loss)# Train Summariestrain_summary_op tf.summary.merge([loss_summary, grad_summaries_merged])train_summary_dir os.path.join(out_dir, summaries, train)train_summary_writer tf.summary.FileWriter(train_summary_dir, sess.graph)# Inference summariesinference_summary_op tf.summary.merge([loss_summary])inference_summary_dir os.path.join(out_dir, summaries, inference)inference_summary_writer tf.summary.FileWriter(inference_summary_dir, sess.graph)sess.run(tf.global_variables_initializer())saver tf.train.Saver()for epoch_i in range(num_epochs):#将数据集分成训练集和测试集随机种子不固定train_X,test_X, train_y, test_y train_test_split(features, targets_values, test_size 0.2, random_state 0) train_batches get_batches(train_X, train_y, batch_size)test_batches get_batches(test_X, test_y, batch_size)#训练的迭代保存训练损失for batch_i in range(len(train_X) // batch_size):x, y next(train_batches)categories np.zeros([batch_size, 18])for i in range(batch_size):categories[i] x.take(6,1)[i]titles np.zeros([batch_size, sentences_size])for i in range(batch_size):titles[i] x.take(5,1)[i]feed {uid: np.reshape(x.take(0,1), [batch_size, 1]),user_gender: np.reshape(x.take(2,1), [batch_size, 1]),user_age: np.reshape(x.take(3,1), [batch_size, 1]),user_job: np.reshape(x.take(4,1), [batch_size, 1]),movie_id: np.reshape(x.take(1,1), [batch_size, 1]),movie_categories: categories, #x.take(6,1)movie_titles: titles, #x.take(5,1)targets: np.reshape(y, [batch_size, 1]),dropout_keep_prob: dropout_keep, #dropout_keeplr: learning_rate}step, train_loss, summaries, _ sess.run([global_step, loss, train_summary_op, train_op], feed) #costlosses[train].append(train_loss)train_summary_writer.add_summary(summaries, step) ## Show every show_every_n_batches batchesif (epoch_i * (len(train_X) // batch_size) batch_i) % show_every_n_batches 0:time_str datetime.datetime.now().isoformat()print({}: Epoch {:3} Batch {:4}/{} train_loss {:.3f}.format(time_str,epoch_i,batch_i,(len(train_X) // batch_size),train_loss))#使用测试数据的迭代for batch_i in range(len(test_X) // batch_size):x, y next(test_batches)categories np.zeros([batch_size, 18])for i in range(batch_size):categories[i] x.take(6,1)[i]titles np.zeros([batch_size, sentences_size])for i in range(batch_size):titles[i] x.take(5,1)[i]feed {uid: np.reshape(x.take(0,1), [batch_size, 1]),user_gender: np.reshape(x.take(2,1), [batch_size, 1]),user_age: np.reshape(x.take(3,1), [batch_size, 1]),user_job: np.reshape(x.take(4,1), [batch_size, 1]),movie_id: np.reshape(x.take(1,1), [batch_size, 1]),movie_categories: categories, #x.take(6,1)movie_titles: titles, #x.take(5,1)targets: np.reshape(y, [batch_size, 1]),dropout_keep_prob: 1,lr: learning_rate}step, test_loss, summaries sess.run([global_step, loss, inference_summary_op], feed) #cost#保存测试损失losses[test].append(test_loss)inference_summary_writer.add_summary(summaries, step) #time_str datetime.datetime.now().isoformat()if (epoch_i * (len(test_X) // batch_size) batch_i) % show_every_n_batches 0:print({}: Epoch {:3} Batch {:4}/{} test_loss {:.3f}.format(time_str,epoch_i,batch_i,(len(test_X) // batch_size),test_loss))# Save Modelsaver.save(sess, save_dir) #, global_stepepoch_iprint(Model Trained and Saved)
在 TensorBoard 中查看可视化结果 获取 Tensors
def get_tensors(loaded_graph):uid loaded_graph.get_tensor_by_name(uid:0)user_gender loaded_graph.get_tensor_by_name(user_gender:0)user_age loaded_graph.get_tensor_by_name(user_age:0)user_job loaded_graph.get_tensor_by_name(user_job:0)movie_id loaded_graph.get_tensor_by_name(movie_id:0)movie_categories loaded_graph.get_tensor_by_name(movie_categories:0)movie_titles loaded_graph.get_tensor_by_name(movie_titles:0)targets loaded_graph.get_tensor_by_name(targets:0)dropout_keep_prob loaded_graph.get_tensor_by_name(dropout_keep_prob:0)lr loaded_graph.get_tensor_by_name(LearningRate:0)#两种不同计算预测评分的方案使用不同的name获取tensor inference
# inference loaded_graph.get_tensor_by_name(inference/inference/BiasAdd:0)inference loaded_graph.get_tensor_by_name(inference/MatMul:0)#movie_combine_layer_flat loaded_graph.get_tensor_by_name(movie_fc/Reshape:0)user_combine_layer_flat loaded_graph.get_tensor_by_name(user_fc/Reshape:0)return uid, user_gender, user_age, user_job, movie_id, movie_categories, movie_titles, targets, lr, dropout_keep_prob, inference, movie_combine_layer_flat, user_combine_layer_flat
指定用户和电影进行评分 这部分就是对网络做正向传播计算得到预测的评分
def rating_movie(user_id_val, movie_id_val):loaded_graph tf.Graph() #with tf.Session(graphloaded_graph) as sess: ## Load saved modelloader tf.train.import_meta_graph(load_dir .meta)loader.restore(sess, load_dir)# Get Tensors from loaded modeluid, user_gender, user_age, user_job, movie_id, movie_categories, movie_titles, targets, lr, dropout_keep_prob, inference,_, __ get_tensors(loaded_graph) #loaded_graphcategories np.zeros([1, 18])categories[0] movies.values[movieid2idx[movie_id_val]][2]titles np.zeros([1, sentences_size])titles[0] movies.values[movieid2idx[movie_id_val]][1]feed {uid: np.reshape(users.values[user_id_val-1][0], [1, 1]),user_gender: np.reshape(users.values[user_id_val-1][1], [1, 1]),user_age: np.reshape(users.values[user_id_val-1][2], [1, 1]),user_job: np.reshape(users.values[user_id_val-1][3], [1, 1]),movie_id: np.reshape(movies.values[movieid2idx[movie_id_val]][0], [1, 1]),movie_categories: categories, #x.take(6,1)movie_titles: titles, #x.take(5,1)dropout_keep_prob: 1}# Get Predictioninference_val sess.run([inference], feed) return (inference_val) 生成Movie特征矩阵 将训练好的电影特征组合成电影特征矩阵并保存到本地
loaded_graph tf.Graph() #
movie_matrics []
with tf.Session(graphloaded_graph) as sess: ## Load saved modelloader tf.train.import_meta_graph(load_dir .meta)loader.restore(sess, load_dir)# Get Tensors from loaded modeluid, user_gender, user_age, user_job, movie_id, movie_categories, movie_titles, targets, lr, dropout_keep_prob, _, movie_combine_layer_flat, __ get_tensors(loaded_graph) #loaded_graphfor item in movies.values:categories np.zeros([1, 18])categories[0] item.take(2)titles np.zeros([1, sentences_size])titles[0] item.take(1)feed {movie_id: np.reshape(item.take(0), [1, 1]),movie_categories: categories, #x.take(6,1)movie_titles: titles, #x.take(5,1)dropout_keep_prob: 1}movie_combine_layer_flat_val sess.run([movie_combine_layer_flat], feed) movie_matrics.append(movie_combine_layer_flat_val)pickle.dump((np.array(movie_matrics).reshape(-1, 200)), open(movie_matrics.p, wb))
movie_matrics pickle.load(open(movie_matrics.p, moderb))
生成User特征矩阵 将训练好的用户特征组合成用户特征矩阵并保存到本地
loaded_graph tf.Graph() #
users_matrics []
with tf.Session(graphloaded_graph) as sess: ## Load saved modelloader tf.train.import_meta_graph(load_dir .meta)loader.restore(sess, load_dir)# Get Tensors from loaded modeluid, user_gender, user_age, user_job, movie_id, movie_categories, movie_titles, targets, lr, dropout_keep_prob, _, __,user_combine_layer_flat get_tensors(loaded_graph) #loaded_graphfor item in users.values:feed {uid: np.reshape(item.take(0), [1, 1]),user_gender: np.reshape(item.take(1), [1, 1]),user_age: np.reshape(item.take(2), [1, 1]),user_job: np.reshape(item.take(3), [1, 1]),dropout_keep_prob: 1}user_combine_layer_flat_val sess.run([user_combine_layer_flat], feed) users_matrics.append(user_combine_layer_flat_val)pickle.dump((np.array(users_matrics).reshape(-1, 200)), open(users_matrics.p, wb))
users_matrics pickle.load(open(users_matrics.p, moderb))
开始推荐电影 使用生产的用户特征矩阵和电影特征矩阵做电影推荐
推荐同类型的电影 思路是计算当前看的电影特征向量与整个电影特征矩阵的余弦相似度取相似度最大的top_k个这里加了些随机选择在里面保证每次的推荐稍稍有些不同。
def recommend_same_type_movie(movie_id_val, top_k 20):loaded_graph tf.Graph() #with tf.Session(graphloaded_graph) as sess: ## Load saved modelloader tf.train.import_meta_graph(load_dir .meta)loader.restore(sess, load_dir)norm_movie_matrics tf.sqrt(tf.reduce_sum(tf.square(movie_matrics), 1, keep_dimsTrue))normalized_movie_matrics movie_matrics / norm_movie_matrics#推荐同类型的电影probs_embeddings (movie_matrics[movieid2idx[movie_id_val]]).reshape([1, 200])probs_similarity tf.matmul(probs_embeddings, tf.transpose(normalized_movie_matrics))sim (probs_similarity.eval())# results (-sim[0]).argsort()[0:top_k]# print(results)print(您看的电影是{}.format(movies_orig[movieid2idx[movie_id_val]]))print(以下是给您的推荐)p np.squeeze(sim)p[np.argsort(p)[:-top_k]] 0p p / np.sum(p)results set()while len(results) ! 5:c np.random.choice(3883, 1, pp)[0]results.add(c)for val in (results):print(val)print(movies_orig[val])return results 推荐您喜欢的电影 思路是使用用户特征向量与电影特征矩阵计算所有电影的评分取评分最高的top_k个同样加了些随机选择部分。
def recommend_your_favorite_movie(user_id_val, top_k 10):loaded_graph tf.Graph() #with tf.Session(graphloaded_graph) as sess: ## Load saved modelloader tf.train.import_meta_graph(load_dir .meta)loader.restore(sess, load_dir)#推荐您喜欢的电影probs_embeddings (users_matrics[user_id_val-1]).reshape([1, 200])probs_similarity tf.matmul(probs_embeddings, tf.transpose(movie_matrics))sim (probs_similarity.eval())# print(sim.shape)# results (-sim[0]).argsort()[0:top_k]# print(results)# sim_norm probs_norm_similarity.eval()# print((-sim_norm[0]).argsort()[0:top_k])print(以下是给您的推荐)p np.squeeze(sim)p[np.argsort(p)[:-top_k]] 0p p / np.sum(p)results set()while len(results) ! 5:c np.random.choice(3883, 1, pp)[0]results.add(c)for val in (results):print(val)print(movies_orig[val])return results
看过这个电影的人还看了喜欢哪些电影 首先选出喜欢某个电影的top_k个人得到这几个人的用户特征向量。 然后计算这几个人对所有电影的评分 选择每个人评分最高的电影作为推荐 同样加入了随机选择
import randomdef recommend_other_favorite_movie(movie_id_val, top_k 20):loaded_graph tf.Graph() #with tf.Session(graphloaded_graph) as sess: ## Load saved modelloader tf.train.import_meta_graph(load_dir .meta)loader.restore(sess, load_dir)probs_movie_embeddings (movie_matrics[movieid2idx[movie_id_val]]).reshape([1, 200])probs_user_favorite_similarity tf.matmul(probs_movie_embeddings, tf.transpose(users_matrics))favorite_user_id np.argsort(probs_user_favorite_similarity.eval())[0][-top_k:]# print(normalized_users_matrics.eval().shape)# print(probs_user_favorite_similarity.eval()[0][favorite_user_id])# print(favorite_user_id.shape)print(您看的电影是{}.format(movies_orig[movieid2idx[movie_id_val]]))print(喜欢看这个电影的人是{}.format(users_orig[favorite_user_id-1]))probs_users_embeddings (users_matrics[favorite_user_id-1]).reshape([-1, 200])probs_similarity tf.matmul(probs_users_embeddings, tf.transpose(movie_matrics))sim (probs_similarity.eval())# results (-sim[0]).argsort()[0:top_k]# print(results)# print(sim.shape)# print(np.argmax(sim, 1))p np.argmax(sim, 1)print(喜欢看这个电影的人还喜欢看)results set()while len(results) ! 5:c p[random.randrange(top_k)]results.add(c)for val in (results):print(val)print(movies_orig[val])return results 结论 以上就是实现的常用的推荐功能将网络模型作为回归问题进行训练得到训练好的用户特征矩阵和电影特征矩阵进行推荐。
任何程序错误以及技术疑问或需要解答的请添加
