个人做的网站有什么危险,哪个网站做自考题目免费,phpcms 后台修改修改网站备案号,上海人才服务网分类目录#xff1a;《深入浅出Pytorch函数》总目录 torch.nn.init模块中的所有函数都用于初始化神经网络参数#xff0c;因此它们都在torc.no_grad()模式下运行#xff0c;autograd不会将其考虑在内。
根据Glorot, X.和Bengio, Y.在《Understanding the difficulty of tra…分类目录《深入浅出Pytorch函数》总目录 torch.nn.init模块中的所有函数都用于初始化神经网络参数因此它们都在torc.no_grad()模式下运行autograd不会将其考虑在内。
根据Glorot, X.和Bengio, Y.在《Understanding the difficulty of training deep feedforward neural networks》中描述的方法用一个均匀分布生成值填充输入的张量或变量。结果张量中的值采样自 U ( − a , a ) U(-a, a) U(−a,a)其中 a gain × 6 fan_in fan_put a\text{gain}\times\sqrt{\frac{6}{\text{fan\_in}\text{fan\_put}}} again×fan_infan_put6
这种方法也被称为Glorot initialization。
语法
torch.nn.init.xavier_uniform_(tensor, gain1)参数
tensor[Tensor] 一个 N N N维张量torch.Tensorgain [float] 可选的缩放因子
返回值
一个torch.Tensor
实例
w torch.empty(3, 5)
nn.init.xavier_uniform_(w, gainnn.init.calculate_gain(relu))函数实现
def xavier_uniform_(tensor: Tensor, gain: float 1.) - Tensor:rFills the input Tensor with values according to the methoddescribed in Understanding the difficulty of training deep feedforwardneural networks - Glorot, X. Bengio, Y. (2010), using a uniformdistribution. The resulting tensor will have values sampled from:math:\mathcal{U}(-a, a) where.. math::a \text{gain} \times \sqrt{\frac{6}{\text{fan\_in} \text{fan\_out}}}Also known as Glorot initialization.Args:tensor: an n-dimensional torch.Tensorgain: an optional scaling factorExamples: w torch.empty(3, 5) nn.init.xavier_uniform_(w, gainnn.init.calculate_gain(relu))fan_in, fan_out _calculate_fan_in_and_fan_out(tensor)std gain * math.sqrt(2.0 / float(fan_in fan_out))a math.sqrt(3.0) * std # Calculate uniform bounds from standard deviationreturn _no_grad_uniform_(tensor, -a, a)
