我想知道为什么这个逻辑模型每次都会输出权重和偏差nan。有什么可能吗?(仅供参考/训练集: 2082行×91列)
import tensorflow as tf
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
features = stock_copy2.iloc[:,:-1]
target = stock_copy2.iloc[:,-1]
x_train, x_test, y_train, y_test = \
train_test_split(features, target, test_size = 0.3, random_state = 1)
tf.random.set_seed(2020)
W = tf.Variable(tf.random.normal([91, 1], mean=0.0))
b = tf.Variable(tf.random.normal([1], mean=0.0))
# Learning Rate
learning_rate = 0.01
# Hypothesis and Prediction Function
def predict(X):
z = tf.matmul(X, W) + b
hypothesis = 1 / (1 + tf.exp(-z))
return hypothesis
# Training
for i in range(2000+1):
with tf.GradientTape() as tape:
hypothesis = predict(x_train)
cost = tf.reduce_mean(-tf.reduce_sum(y_train*tf.math.log(hypothesis) + (1-y_train)*tf.math.log(1-hypothesis)))
W_grad, b_grad = tape.gradient(cost, [W, b])
W.assign_sub(learning_rate * W_grad)
b.assign_sub(learning_rate * b_grad)
if i % 400 == 0:
print(">>> #%s \n Weights: \n%s \n Bias: \n%s \n cost: %s\n" % (i, W.numpy(), b.numpy(), cost.numpy()))
除了我设置的第一个随机权重和偏差外,输出是这样的。
Weights:
[[nan]
[nan]
[nan]
[nan]
[nan]
[nan] ....
发布于 2020-10-23 10:02:49
使用固定的学习率进行优化通常是一个坏主意,因为它在最好的情况下会产生次优的收敛结果,但通常会导致模型发散。我使用RMSprop优化器让你的代码收敛(在随机数据上),它会自动调整你的学习速度。
您可以在训练之前定义keras优化器,如下所示
optimizer = tf.keras.optimizers.RMSprop(learning_rate=0.001, rho=0.9)
然后用它来应用你的胶带渐变:
optimizer.apply_gradients(zip([W_grad, b_grad], [W,b]))
而不是减去梯度。有关可用的优化器模块的更多信息,请查看https://www.tensorflow.org/api_docs/python/tf/keras/optimizers!
https://stackoverflow.com/questions/64496678
复制