Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dropout和batchnorm层已经传了training,outputs._uses_learning_phase = training is not None的作用是什么? #506

Open
Daemoonn opened this issue Dec 13, 2022 · 0 comments
Labels

Comments

@Daemoonn
Copy link

Describe the question(问题描述)
既然DNN中的bn_layers和dropout_layers已经传入了training状态参数来区分是在训练还是在预测阶段,手动设置_uses_learning_phase的作用是什么?
我的tf版本2.10.0,应该是执行outputs._uses_learning_phase = training is not None

attention_score = self.local_att([queries, keys], training=training)
outputs = tf.transpose(attention_score, (0, 2, 1))
if self.weight_normalization:
paddings = tf.ones_like(outputs) * (-2 ** 32 + 1)
else:
paddings = tf.zeros_like(outputs)
outputs = tf.where(key_masks, outputs, paddings)
if self.weight_normalization:
outputs = softmax(outputs)
if not self.return_score:
outputs = tf.matmul(outputs, keys)
if tf.__version__ < '1.13.0':
outputs._uses_learning_phase = attention_score._uses_learning_phase
else:
outputs._uses_learning_phase = training is not None
return outputs

在求attention_score的时候,不是已经把training状态传给DNN里的dropout和batchnorm层了吗?

attention_score = self.local_att([queries, keys], training=training)

att_out = self.dnn(att_input, training=training)

fc = self.bn_layers[i](fc, training=training)

fc = self.dropout_layers[i](fc, training=training)

为什么还需要对_uses_learning_phase手动设置?

if tf.__version__ < '1.13.0':
outputs._uses_learning_phase = attention_score._uses_learning_phase
else:
outputs._uses_learning_phase = training is not None

这里手动设置_uses_learning_phase是否发挥实际作用?去掉会怎么样?对于outputs这个tf.Tensor来说,好像是没有_uses_learning_phase这个类属性,这么写好像是临时新建个了_uses_learning_phase属性,然后赋值成xxx

outputs._uses_learning_phase = xxx

求大佬指点

Additional context
Add any other context about the problem here.

Operating environment(运行环境):

  • python version 3.9.12
  • tensorflow version 2.10.0
  • deepctr version 0.9.3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant