torch.nn.functional.conv1d(input, weight, bias=None, stride=1, padding=0, dilation=1, groups=1) → Tensor
Applies a 1D convolution over an input signal composed of several input planes.
This operator supports TensorFloat32.
See Conv1d
for details and output shape.
Note
In some circumstances when using the CUDA backend with CuDNN, this operator may select a nondeterministic algorithm to increase performance. If this is undesirable, you can try to make the operation deterministic (potentially at a performance cost) by setting torch.backends.cudnn.deterministic =
True
. Please see the notes on Reproducibility for background.
None
(sW,)
. Default: 1(padW,)
. Default: 0(dW,)
. Default: 1Examples:
>>> filters = torch.randn(33, 16, 3) >>> inputs = torch.randn(20, 16, 50) >>> F.conv1d(inputs, filters)
torch.nn.functional.conv2d(input, weight, bias=None, stride=1, padding=0, dilation=1, groups=1) → Tensor
Applies a 2D convolution over an input image composed of several input planes.
This operator supports TensorFloat32.
See Conv2d
for details and output shape.
Note
In some circumstances when using the CUDA backend with CuDNN, this operator may select a nondeterministic algorithm to increase performance. If this is undesirable, you can try to make the operation deterministic (potentially at a performance cost) by setting torch.backends.cudnn.deterministic =
True
. Please see the notes on Reproducibility for background.
None
(sH, sW)
. Default: 1(padH, padW)
. Default: 0(dH, dW)
. Default: 1Examples:
>>> # With square kernels and equal stride >>> filters = torch.randn(8,4,3,3) >>> inputs = torch.randn(1,4,5,5) >>> F.conv2d(inputs, filters, padding=1)
torch.nn.functional.conv3d(input, weight, bias=None, stride=1, padding=0, dilation=1, groups=1) → Tensor
Applies a 3D convolution over an input image composed of several input planes.
This operator supports TensorFloat32.
See Conv3d
for details and output shape.
Note
In some circumstances when using the CUDA backend with CuDNN, this operator may select a nondeterministic algorithm to increase performance. If this is undesirable, you can try to make the operation deterministic (potentially at a performance cost) by setting torch.backends.cudnn.deterministic =
True
. Please see the notes on Reproducibility for background.
(sT, sH, sW)
. Default: 1(padT, padH, padW)
. Default: 0(dT, dH, dW)
. Default: 1Examples:
>>> filters = torch.randn(33, 16, 3, 3, 3) >>> inputs = torch.randn(20, 16, 50, 10, 20) >>> F.conv3d(inputs, filters)
torch.nn.functional.conv_transpose1d(input, weight, bias=None, stride=1, padding=0, output_padding=0, groups=1, dilation=1) → Tensor
Applies a 1D transposed convolution operator over an input signal composed of several input planes, sometimes also called “deconvolution”.
This operator supports TensorFloat32.
See ConvTranspose1d
for details and output shape.
Note
In some circumstances when using the CUDA backend with CuDNN, this operator may select a nondeterministic algorithm to increase performance. If this is undesirable, you can try to make the operation deterministic (potentially at a performance cost) by setting torch.backends.cudnn.deterministic =
True
. Please see the notes on Reproducibility for background.
(sW,)
. Default: 1dilation * (kernel_size - 1) - padding
zero-padding will be added to both sides of each dimension in the input. Can be a single number or a tuple (padW,)
. Default: 0(out_padW)
. Default: 0(dW,)
. Default: 1Examples:
>>> inputs = torch.randn(20, 16, 50) >>> weights = torch.randn(16, 33, 5) >>> F.conv_transpose1d(inputs, weights)
torch.nn.functional.conv_transpose2d(input, weight, bias=None, stride=1, padding=0, output_padding=0, groups=1, dilation=1) → Tensor
Applies a 2D transposed convolution operator over an input image composed of several input planes, sometimes also called “deconvolution”.
This operator supports TensorFloat32.
See ConvTranspose2d
for details and output shape.
Note
In some circumstances when using the CUDA backend with CuDNN, this operator may select a nondeterministic algorithm to increase performance. If this is undesirable, you can try to make the operation deterministic (potentially at a performance cost) by setting torch.backends.cudnn.deterministic =
True
. Please see the notes on Reproducibility for background.
(sH, sW)
. Default: 1dilation * (kernel_size - 1) - padding
zero-padding will be added to both sides of each dimension in the input. Can be a single number or a tuple (padH, padW)
. Default: 0(out_padH, out_padW)
. Default: 0(dH, dW)
. Default: 1Examples:
>>> # With square kernels and equal stride >>> inputs = torch.randn(1, 4, 5, 5) >>> weights = torch.randn(4, 8, 3, 3) >>> F.conv_transpose2d(inputs, weights, padding=1)
torch.nn.functional.conv_transpose3d(input, weight, bias=None, stride=1, padding=0, output_padding=0, groups=1, dilation=1) → Tensor
Applies a 3D transposed convolution operator over an input image composed of several input planes, sometimes also called “deconvolution”
This operator supports TensorFloat32.
See ConvTranspose3d
for details and output shape.
Note
In some circumstances when using the CUDA backend with CuDNN, this operator may select a nondeterministic algorithm to increase performance. If this is undesirable, you can try to make the operation deterministic (potentially at a performance cost) by setting torch.backends.cudnn.deterministic =
True
. Please see the notes on Reproducibility for background.
(sT, sH, sW)
. Default: 1dilation * (kernel_size - 1) - padding
zero-padding will be added to both sides of each dimension in the input. Can be a single number or a tuple (padT, padH, padW)
. Default: 0(out_padT, out_padH, out_padW)
. Default: 0(dT, dH, dW)
. Default: 1Examples:
>>> inputs = torch.randn(20, 16, 50, 10, 20) >>> weights = torch.randn(16, 33, 3, 3, 3) >>> F.conv_transpose3d(inputs, weights)
torch.nn.functional.unfold(input, kernel_size, dilation=1, padding=0, stride=1)
[source]
Extracts sliding local blocks from an batched input tensor.
Warning
Currently, only 4-D input tensors (batched image-like tensors) are supported.
Warning
More than one element of the unfolded tensor may refer to a single memory location. As a result, in-place operations (especially ones that are vectorized) may result in incorrect behavior. If you need to write to the tensor, please clone it first.
See torch.nn.Unfold
for details
torch.nn.functional.fold(input, output_size, kernel_size, dilation=1, padding=0, stride=1)
[source]
Combines an array of sliding local blocks into a large containing tensor.
Warning
Currently, only 3-D output tensors (unfolded batched image-like tensors) are supported.
See torch.nn.Fold
for details
torch.nn.functional.avg_pool1d(input, kernel_size, stride=None, padding=0, ceil_mode=False, count_include_pad=True) → Tensor
Applies a 1D average pooling over an input signal composed of several input planes.
See AvgPool1d
for details and output shape.
(kW,)
(sW,)
. Default: kernel_size
(padW,)
. Default: 0ceil
instead of floor
to compute the output shape. Default: False
True
Examples:
>>> # pool of square window of size=3, stride=2 >>> input = torch.tensor([[[1, 2, 3, 4, 5, 6, 7]]], dtype=torch.float32) >>> F.avg_pool1d(input, kernel_size=3, stride=2) tensor([[[ 2., 4., 6.]]])
torch.nn.functional.avg_pool2d(input, kernel_size, stride=None, padding=0, ceil_mode=False, count_include_pad=True, divisor_override=None) → Tensor
Applies 2D average-pooling operation in $kH \times kW$ regions by step size $sH \times sW$ steps. The number of output features is equal to the number of input planes.
See AvgPool2d
for details and output shape.
(kH, kW)
(sH, sW)
. Default: kernel_size
(padH, padW)
. Default: 0ceil
instead of floor
in the formula to compute the output shape. Default: False
True
torch.nn.functional.avg_pool3d(input, kernel_size, stride=None, padding=0, ceil_mode=False, count_include_pad=True, divisor_override=None) → Tensor
Applies 3D average-pooling operation in $kT \times kH \times kW$ regions by step size $sT \times sH \times sW$ steps. The number of output features is equal to $\lfloor\frac{\text{input planes}}{sT}\rfloor$ .
See AvgPool3d
for details and output shape.
(kT, kH, kW)
(sT, sH, sW)
. Default: kernel_size
(padT, padH, padW)
, Default: 0ceil
instead of floor
in the formula to compute the output shapetorch.nn.functional.max_pool1d(*args, **kwargs)
Applies a 1D max pooling over an input signal composed of several input planes.
See MaxPool1d
for details.
torch.nn.functional.max_pool2d(*args, **kwargs)
Applies a 2D max pooling over an input signal composed of several input planes.
See MaxPool2d
for details.
torch.nn.functional.max_pool3d(*args, **kwargs)
Applies a 3D max pooling over an input signal composed of several input planes.
See MaxPool3d
for details.
torch.nn.functional.max_unpool1d(input, indices, kernel_size, stride=None, padding=0, output_size=None)
[source]
Computes a partial inverse of MaxPool1d
.
See MaxUnpool1d
for details.
torch.nn.functional.max_unpool2d(input, indices, kernel_size, stride=None, padding=0, output_size=None)
[source]
Computes a partial inverse of MaxPool2d
.
See MaxUnpool2d
for details.
torch.nn.functional.max_unpool3d(input, indices, kernel_size, stride=None, padding=0, output_size=None)
[source]
Computes a partial inverse of MaxPool3d
.
See MaxUnpool3d
for details.
torch.nn.functional.lp_pool1d(input, norm_type, kernel_size, stride=None, ceil_mode=False)
[source]
Applies a 1D power-average pooling over an input signal composed of several input planes. If the sum of all inputs to the power of p
is zero, the gradient is set to zero as well.
See LPPool1d
for details.
torch.nn.functional.lp_pool2d(input, norm_type, kernel_size, stride=None, ceil_mode=False)
[source]
Applies a 2D power-average pooling over an input signal composed of several input planes. If the sum of all inputs to the power of p
is zero, the gradient is set to zero as well.
See LPPool2d
for details.
torch.nn.functional.adaptive_max_pool1d(*args, **kwargs)
Applies a 1D adaptive max pooling over an input signal composed of several input planes.
See AdaptiveMaxPool1d
for details and output shape.
False
torch.nn.functional.adaptive_max_pool2d(*args, **kwargs)
Applies a 2D adaptive max pooling over an input signal composed of several input planes.
See AdaptiveMaxPool2d
for details and output shape.
False
torch.nn.functional.adaptive_max_pool3d(*args, **kwargs)
Applies a 3D adaptive max pooling over an input signal composed of several input planes.
See AdaptiveMaxPool3d
for details and output shape.
False
torch.nn.functional.adaptive_avg_pool1d(input, output_size) → Tensor
Applies a 1D adaptive average pooling over an input signal composed of several input planes.
See AdaptiveAvgPool1d
for details and output shape.
output_size – the target output size (single integer)
torch.nn.functional.adaptive_avg_pool2d(input, output_size)
[source]
Applies a 2D adaptive average pooling over an input signal composed of several input planes.
See AdaptiveAvgPool2d
for details and output shape.
output_size – the target output size (single integer or double-integer tuple)
torch.nn.functional.adaptive_avg_pool3d(input, output_size)
[source]
Applies a 3D adaptive average pooling over an input signal composed of several input planes.
See AdaptiveAvgPool3d
for details and output shape.
output_size – the target output size (single integer or triple-integer tuple)
torch.nn.functional.threshold(input, threshold, value, inplace=False)
Thresholds each element of the input Tensor.
See Threshold
for more details.
torch.nn.functional.threshold_(input, threshold, value) → Tensor
In-place version of threshold()
.
torch.nn.functional.relu(input, inplace=False) → Tensor
[source]
Applies the rectified linear unit function element-wise. See ReLU
for more details.
torch.nn.functional.relu_(input) → Tensor
In-place version of relu()
.
torch.nn.functional.hardtanh(input, min_val=-1., max_val=1., inplace=False) → Tensor
[source]
Applies the HardTanh function element-wise. See Hardtanh
for more details.
torch.nn.functional.hardtanh_(input, min_val=-1., max_val=1.) → Tensor
In-place version of hardtanh()
.
torch.nn.functional.hardswish(input: torch.Tensor, inplace: bool = False) → torch.Tensor
[source]
Applies the hardswish function, element-wise, as described in the paper:
See Hardswish
for more details.
torch.nn.functional.relu6(input, inplace=False) → Tensor
[source]
Applies the element-wise function $\text{ReLU6}(x) = \min(\max(0,x), 6)$ .
See ReLU6
for more details.
torch.nn.functional.elu(input, alpha=1.0, inplace=False)
[source]
Applies element-wise, $\text{ELU}(x) = \max(0,x) + \min(0, \alpha * (\exp(x) - 1))$ .
See ELU
for more details.
torch.nn.functional.elu_(input, alpha=1.) → Tensor
In-place version of elu()
.
torch.nn.functional.selu(input, inplace=False) → Tensor
[source]
Applies element-wise, $\text{SELU}(x) = scale * (\max(0,x) + \min(0, \alpha * (\exp(x) - 1)))$ , with $\alpha=1.6732632423543772848170429916717$ and $scale=1.0507009873554804934193349852946$ .
See SELU
for more details.
torch.nn.functional.celu(input, alpha=1., inplace=False) → Tensor
[source]
Applies element-wise, $\text{CELU}(x) = \max(0,x) + \min(0, \alpha * (\exp(x/\alpha) - 1))$ .
See CELU
for more details.
torch.nn.functional.leaky_relu(input, negative_slope=0.01, inplace=False) → Tensor
[source]
Applies element-wise, $\text{LeakyReLU}(x) = \max(0, x) + \text{negative\_slope} * \min(0, x)$
See LeakyReLU
for more details.
torch.nn.functional.leaky_relu_(input, negative_slope=0.01) → Tensor
In-place version of leaky_relu()
.
torch.nn.functional.prelu(input, weight) → Tensor
[source]
Applies element-wise the function $\text{PReLU}(x) = \max(0,x) + \text{weight} * \min(0,x)$ where weight is a learnable parameter.
See PReLU
for more details.
torch.nn.functional.rrelu(input, lower=1./8, upper=1./3, training=False, inplace=False) → Tensor
[source]
Randomized leaky ReLU.
See RReLU
for more details.
torch.nn.functional.rrelu_(input, lower=1./8, upper=1./3, training=False) → Tensor
In-place version of rrelu()
.
torch.nn.functional.glu(input, dim=-1) → Tensor
[source]
The gated linear unit. Computes:
where input
is split in half along dim
to form a
and b
, $\sigma$ is the sigmoid function and $\otimes$ is the element-wise product between matrices.
torch.nn.functional.gelu(input) → Tensor
[source]
Applies element-wise the function $\text{GELU}(x) = x * \Phi(x)$
where $\Phi(x)$ is the Cumulative Distribution Function for Gaussian Distribution.
torch.nn.functional.logsigmoid(input) → Tensor
Applies element-wise $\text{LogSigmoid}(x_i) = \log \left(\frac{1}{1 + \exp(-x_i)}\right)$
See LogSigmoid
for more details.
torch.nn.functional.hardshrink(input, lambd=0.5) → Tensor
[source]
Applies the hard shrinkage function element-wise
See Hardshrink
for more details.
torch.nn.functional.tanhshrink(input) → Tensor
[source]
Applies element-wise, $\text{Tanhshrink}(x) = x - \text{Tanh}(x)$
See Tanhshrink
for more details.
torch.nn.functional.softsign(input) → Tensor
[source]
Applies element-wise, the function $\text{SoftSign}(x) = \frac{x}{1 + |x|}$
See Softsign
for more details.
torch.nn.functional.softplus(input, beta=1, threshold=20) → Tensor
Applies element-wise, the function $\text{Softplus}(x) = \frac{1}{\beta} * \log(1 + \exp(\beta * x))$ .
For numerical stability the implementation reverts to the linear function when $input \times \beta > threshold$ .
See Softplus
for more details.
torch.nn.functional.softmin(input, dim=None, _stacklevel=3, dtype=None)
[source]
Applies a softmin function.
Note that $\text{Softmin}(x) = \text{Softmax}(-x)$ . See softmax definition for mathematical formula.
See Softmin
for more details.
torch.dtype
, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype
before the operation is performed. This is useful for preventing data type overflows. Default: None.torch.nn.functional.softmax(input, dim=None, _stacklevel=3, dtype=None)
[source]
Applies a softmax function.
Softmax is defined as:
$\text{Softmax}(x_{i}) = \frac{\exp(x_i)}{\sum_j \exp(x_j)}$
It is applied to all slices along dim, and will re-scale them so that the elements lie in the range [0, 1]
and sum to 1.
See Softmax
for more details.
torch.dtype
, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype
before the operation is performed. This is useful for preventing data type overflows. Default: None.Note
This function doesn’t work directly with NLLLoss, which expects the Log to be computed between the Softmax and itself. Use log_softmax instead (it’s faster and has better numerical properties).
torch.nn.functional.softshrink(input, lambd=0.5) → Tensor
Applies the soft shrinkage function elementwise
See Softshrink
for more details.
torch.nn.functional.gumbel_softmax(logits, tau=1, hard=False, eps=1e-10, dim=-1)
[source]
Samples from the Gumbel-Softmax distribution (Link 1 Link 2) and optionally discretizes.
[…, num_features]
unnormalized log probabilitiesTrue
, the returned samples will be discretized as one-hot vectors, but will be differentiated as if it is the soft sample in autogradSampled tensor of same shape as logits
from the Gumbel-Softmax distribution. If hard=True
, the returned samples will be one-hot, otherwise they will be probability distributions that sum to 1 across dim
.
Note
This function is here for legacy reasons, may be removed from nn.Functional in the future.
Note
The main trick for hard
is to do y_hard - y_soft.detach() + y_soft
It achieves two things: - makes the output value exactly one-hot (since we add then subtract y_soft value) - makes the gradient equal to y_soft gradient (since we strip all other gradients)
>>> logits = torch.randn(20, 32) >>> # Sample soft categorical using reparametrization trick: >>> F.gumbel_softmax(logits, tau=1, hard=False) >>> # Sample hard categorical using "Straight-through" trick: >>> F.gumbel_softmax(logits, tau=1, hard=True)
torch.nn.functional.log_softmax(input, dim=None, _stacklevel=3, dtype=None)
[source]
Applies a softmax followed by a logarithm.
While mathematically equivalent to log(softmax(x)), doing these two operations separately is slower, and numerically unstable. This function uses an alternative formulation to compute the output and gradient correctly.
See LogSoftmax
for more details.
torch.dtype
, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype
before the operation is performed. This is useful for preventing data type overflows. Default: None.torch.nn.functional.tanh(input) → Tensor
[source]
Applies element-wise, $\text{Tanh}(x) = \tanh(x) = \frac{\exp(x) - \exp(-x)}{\exp(x) + \exp(-x)}$
See Tanh
for more details.
torch.nn.functional.sigmoid(input) → Tensor
[source]
Applies the element-wise function $\text{Sigmoid}(x) = \frac{1}{1 + \exp(-x)}$
See Sigmoid
for more details.
torch.nn.functional.hardsigmoid(input) → Tensor
[source]
Applies the element-wise function
inplace – If set to True
, will do this operation in-place. Default: False
See Hardsigmoid
for more details.
torch.nn.functional.silu(input, inplace=False)
[source]
Applies the silu function, element-wise.
Note
See Gaussian Error Linear Units (GELUs) where the SiLU (Sigmoid Linear Unit) was originally coined, and see Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning and Swish: a Self-Gated Activation Function where the SiLU was experimented with later.
See SiLU
for more details.
torch.nn.functional.batch_norm(input, running_mean, running_var, weight=None, bias=None, training=False, momentum=0.1, eps=1e-05)
[source]
Applies Batch Normalization for each channel across a batch of data.
See BatchNorm1d
, BatchNorm2d
, BatchNorm3d
for details.
torch.nn.functional.instance_norm(input, running_mean=None, running_var=None, weight=None, bias=None, use_input_stats=True, momentum=0.1, eps=1e-05)
[source]
Applies Instance Normalization for each channel in each data sample in a batch.
See InstanceNorm1d
, InstanceNorm2d
, InstanceNorm3d
for details.
torch.nn.functional.layer_norm(input, normalized_shape, weight=None, bias=None, eps=1e-05)
[source]
Applies Layer Normalization for last certain number of dimensions.
See LayerNorm
for details.
torch.nn.functional.local_response_norm(input, size, alpha=0.0001, beta=0.75, k=1.0)
[source]
Applies local response normalization over an input signal composed of several input planes, where channels occupy the second dimension. Applies normalization across channels.
See LocalResponseNorm
for details.
torch.nn.functional.normalize(input, p=2, dim=1, eps=1e-12, out=None)
[source]
Performs $L_p$ normalization of inputs over specified dimension.
For a tensor input
of sizes $(n_0, ..., n_{dim}, ..., n_k)$ , each $n_{dim}$ -element vector $v$ along dimension dim
is transformed as
With the default arguments it uses the Euclidean norm over vectors along dimension $1$ for normalization.
out
is used, this operation won’t be differentiable.torch.nn.functional.linear(input, weight, bias=None)
[source]
Applies a linear transformation to the incoming data: $y = xA^T + b$ .
This operator supports TensorFloat32.
Shape:
*
means any number of additional dimensionstorch.nn.functional.bilinear(input1, input2, weight, bias=None)
[source]
Applies a bilinear transformation to the incoming data: $y = x_1^T A x_2 + b$
Shape:
torch.nn.functional.dropout(input, p=0.5, training=True, inplace=False)
[source]
During training, randomly zeroes some of the elements of the input tensor with probability p
using samples from a Bernoulli distribution.
See Dropout
for details.
True
. Default: True
True
, will do this operation in-place. Default: False
torch.nn.functional.alpha_dropout(input, p=0.5, training=False, inplace=False)
[source]
Applies alpha dropout to the input.
See AlphaDropout
for details.
torch.nn.functional.feature_alpha_dropout(input, p=0.5, training=False, inplace=False)
[source]
Randomly masks out entire channels (a channel is a feature map, e.g. the $j$ -th channel of the $i$ -th sample in the batch input is a tensor $\text{input}[i, j]$ ) of the input tensor). Instead of setting activations to zero, as in regular Dropout, the activations are set to the negative saturation value of the SELU activation function.
Each element will be masked independently on every forward call with probability p
using samples from a Bernoulli distribution. The elements to be masked are randomized on every forward call, and scaled and shifted to maintain zero mean and unit variance.
See FeatureAlphaDropout
for details.
True
. Default: True
True
, will do this operation in-place. Default: False
torch.nn.functional.dropout2d(input, p=0.5, training=True, inplace=False)
[source]
Randomly zero out entire channels (a channel is a 2D feature map, e.g., the $j$ -th channel of the $i$ -th sample in the batched input is a 2D tensor $\text{input}[i, j]$ ) of the input tensor). Each channel will be zeroed out independently on every forward call with probability p
using samples from a Bernoulli distribution.
See Dropout2d
for details.
True
. Default: True
True
, will do this operation in-place. Default: False
torch.nn.functional.dropout3d(input, p=0.5, training=True, inplace=False)
[source]
Randomly zero out entire channels (a channel is a 3D feature map, e.g., the $j$ -th channel of the $i$ -th sample in the batched input is a 3D tensor $\text{input}[i, j]$ ) of the input tensor). Each channel will be zeroed out independently on every forward call with probability p
using samples from a Bernoulli distribution.
See Dropout3d
for details.
True
. Default: True
True
, will do this operation in-place. Default: False
torch.nn.functional.embedding(input, weight, padding_idx=None, max_norm=None, norm_type=2.0, scale_grad_by_freq=False, sparse=False)
[source]
A simple lookup table that looks up embeddings in a fixed dictionary and size.
This module is often used to retrieve word embeddings using indices. The input to the module is a list of indices, and the embedding matrix, and the output is the corresponding word embeddings.
See torch.nn.Embedding
for more details.
padding_idx
(initialized to zeros) whenever it encounters the index.max_norm
is renormalized to have norm max_norm
. Note: this will modify weight
in-place.max_norm
option. Default 2
.False
.True
, gradient w.r.t. weight
will be a sparse tensor. See Notes under torch.nn.Embedding
for more details regarding sparse gradients.Weight: Embedding matrix of floating point type with shape (V, embedding_dim),
where V = maximum index + 1 and embedding_dim = the embedding size
(*, embedding_dim)
, where *
is the input shapeExamples:
>>> # a batch of 2 samples of 4 indices each >>> input = torch.tensor([[1,2,4,5],[4,3,2,9]]) >>> # an embedding matrix containing 10 tensors of size 3 >>> embedding_matrix = torch.rand(10, 3) >>> F.embedding(input, embedding_matrix) tensor([[[ 0.8490, 0.9625, 0.6753], [ 0.9666, 0.7761, 0.6108], [ 0.6246, 0.9751, 0.3618], [ 0.4161, 0.2419, 0.7383]], [[ 0.6246, 0.9751, 0.3618], [ 0.0237, 0.7794, 0.0528], [ 0.9666, 0.7761, 0.6108], [ 0.3385, 0.8612, 0.1867]]]) >>> # example with padding_idx >>> weights = torch.rand(10, 3) >>> weights[0, :].zero_() >>> embedding_matrix = weights >>> input = torch.tensor([[0,2,0,5]]) >>> F.embedding(input, embedding_matrix, padding_idx=0) tensor([[[ 0.0000, 0.0000, 0.0000], [ 0.5609, 0.5384, 0.8720], [ 0.0000, 0.0000, 0.0000], [ 0.6262, 0.2438, 0.7471]]])
torch.nn.functional.embedding_bag(input, weight, offsets=None, max_norm=None, norm_type=2, scale_grad_by_freq=False, mode='mean', sparse=False, per_sample_weights=None, include_last_offset=False)
[source]
Computes sums, means or maxes of bags
of embeddings, without instantiating the intermediate embeddings.
See torch.nn.EmbeddingBag
for more details.
Note
When using the CUDA backend, this operation may induce nondeterministic behaviour in its backward pass that is not easily switched off. Please see the notes on Reproducibility for background.
input
is 1D. offsets
determines the starting index position of each bag (sequence) in input
.max_norm
is renormalized to have norm max_norm
. Note: this will modify weight
in-place.p
in the p
-norm to compute for the max_norm
option. Default 2
.False
. Note: this option is not supported when mode="max"
."sum"
, "mean"
or "max"
. Specifies the way to reduce the bag. Default: "mean"
True
, gradient w.r.t. weight
will be a sparse tensor. See Notes under torch.nn.Embedding
for more details regarding sparse gradients. Note: this option is not supported when mode="max"
.per_sample_weights
must have exactly the same shape as input and is treated as having the same offsets
, if those are not None.True
, the size of offsets is equal to the number of bags + 1.Shape:
input
(LongTensor) and offsets
(LongTensor, optional)
If input
is 2D of shape (B, N)
,
it will be treated as B
bags (sequences) each of fixed length N
, and this will return B
values aggregated in a way depending on the mode
. offsets
is ignored and required to be None
in this case.
If input
is 1D of shape (N)
,
it will be treated as a concatenation of multiple bags (sequences). offsets
is required to be a 1D tensor containing the starting index positions of each bag in input
. Therefore, for offsets
of shape (B)
, input
will be viewed as having B
bags. Empty bags (i.e., having 0-length) will have returned vectors filled by zeros.
weight
(Tensor): the learnable weights of the module of shape (num_embeddings, embedding_dim)
per_sample_weights
(Tensor, optional). Has the same shape as input
.output
: aggregated embedding values of shape (B, embedding_dim)
Examples:
>>> # an Embedding module containing 10 tensors of size 3 >>> embedding_matrix = torch.rand(10, 3) >>> # a batch of 2 samples of 4 indices each >>> input = torch.tensor([1,2,4,5,4,3,2,9]) >>> offsets = torch.tensor([0,4]) >>> F.embedding_bag(embedding_matrix, input, offsets) tensor([[ 0.3397, 0.3552, 0.5545], [ 0.5893, 0.4386, 0.5882]])
torch.nn.functional.one_hot(tensor, num_classes=-1) → LongTensor
Takes LongTensor with index values of shape (*)
and returns a tensor of shape (*, num_classes)
that have zeros everywhere except where the index of last dimension matches the corresponding value of the input tensor, in which case it will be 1.
See also One-hot on Wikipedia .
LongTensor that has one more dimension with 1 values at the index of last dimension indicated by the input, and 0 everywhere else.
>>> F.one_hot(torch.arange(0, 5) % 3) tensor([[1, 0, 0], [0, 1, 0], [0, 0, 1], [1, 0, 0], [0, 1, 0]]) >>> F.one_hot(torch.arange(0, 5) % 3, num_classes=5) tensor([[1, 0, 0, 0, 0], [0, 1, 0, 0, 0], [0, 0, 1, 0, 0], [1, 0, 0, 0, 0], [0, 1, 0, 0, 0]]) >>> F.one_hot(torch.arange(0, 6).view(3,2) % 3) tensor([[[1, 0, 0], [0, 1, 0]], [[0, 0, 1], [1, 0, 0]], [[0, 1, 0], [0, 0, 1]]])
torch.nn.functional.pairwise_distance(x1, x2, p=2.0, eps=1e-06, keepdim=False)
[source]
See torch.nn.PairwiseDistance
for details
torch.nn.functional.cosine_similarity(x1, x2, dim=1, eps=1e-8) → Tensor
Returns cosine similarity between x1 and x2, computed along dim.
dim
.dim
.Example:
>>> input1 = torch.randn(100, 128) >>> input2 = torch.randn(100, 128) >>> output = F.cosine_similarity(input1, input2) >>> print(output)
torch.nn.functional.pdist(input, p=2) → Tensor
Computes the p-norm distance between every pair of row vectors in the input. This is identical to the upper triangular portion, excluding the diagonal, of torch.norm(input[:, None] - input, dim=2, p=p)
. This function will be faster if the rows are contiguous.
If input has shape $N \times M$ then the output will have shape $\frac{1}{2} N (N - 1)$ .
This function is equivalent to scipy.spatial.distance.pdist(input, ‘minkowski’, p=p)
if $p \in (0, \infty)$ . When $p = 0$ it is equivalent to scipy.spatial.distance.pdist(input, ‘hamming’) * M
. When $p = \infty$ , the closest scipy function is scipy.spatial.distance.pdist(xn, lambda x, y: np.abs(x - y).max())
.
torch.nn.functional.binary_cross_entropy(input, target, weight=None, size_average=None, reduce=None, reduction='mean')
[source]
Function that measures the Binary Cross Entropy between the target and the output.
See BCELoss
for details.
reduction
). By default, the losses are averaged over each loss element in the batch. Note that for some losses, there multiple elements per sample. If the field size_average
is set to False
, the losses are instead summed for each minibatch. Ignored when reduce is False
. Default: True
reduction
). By default, the losses are averaged or summed over observations for each minibatch depending on size_average
. When reduce
is False
, returns a loss per batch element instead and ignores size_average
. Default: True
'none'
| 'mean'
| 'sum'
. 'none'
: no reduction will be applied, 'mean'
: the sum of the output will be divided by the number of elements in the output, 'sum'
: the output will be summed. Note: size_average
and reduce
are in the process of being deprecated, and in the meantime, specifying either of those two args will override reduction
. Default: 'mean'
Examples:
>>> input = torch.randn((3, 2), requires_grad=True) >>> target = torch.rand((3, 2), requires_grad=False) >>> loss = F.binary_cross_entropy(F.sigmoid(input), target) >>> loss.backward()
torch.nn.functional.binary_cross_entropy_with_logits(input, target, weight=None, size_average=None, reduce=None, reduction='mean', pos_weight=None)
[source]
Function that measures Binary Cross Entropy between target and output logits.
See BCEWithLogitsLoss
for details.
reduction
). By default, the losses are averaged over each loss element in the batch. Note that for some losses, there multiple elements per sample. If the field size_average
is set to False
, the losses are instead summed for each minibatch. Ignored when reduce is False
. Default: True
reduction
). By default, the losses are averaged or summed over observations for each minibatch depending on size_average
. When reduce
is False
, returns a loss per batch element instead and ignores size_average
. Default: True
'none'
| 'mean'
| 'sum'
. 'none'
: no reduction will be applied, 'mean'
: the sum of the output will be divided by the number of elements in the output, 'sum'
: the output will be summed. Note: size_average
and reduce
are in the process of being deprecated, and in the meantime, specifying either of those two args will override reduction
. Default: 'mean'
Examples:
>>> input = torch.randn(3, requires_grad=True) >>> target = torch.empty(3).random_(2) >>> loss = F.binary_cross_entropy_with_logits(input, target) >>> loss.backward()
torch.nn.functional.poisson_nll_loss(input, target, log_input=True, full=False, size_average=None, eps=1e-08, reduce=None, reduction='mean')
[source]
Poisson negative log likelihood loss.
See PoissonNLLLoss
for details.
True
the loss is computed as $\exp(\text{input}) - \text{target} * \text{input}$ , if False
then loss is $\text{input} - \text{target} * \log(\text{input}+\text{eps})$ . Default: True
False
$\text{target} * \log(\text{target}) - \text{target} + 0.5 * \log(2 * \pi * \text{target})$ .reduction
). By default, the losses are averaged over each loss element in the batch. Note that for some losses, there multiple elements per sample. If the field size_average
is set to False
, the losses are instead summed for each minibatch. Ignored when reduce is False
. Default: True
log_input`=``False`
. Default: 1e-8reduction
). By default, the losses are averaged or summed over observations for each minibatch depending on size_average
. When reduce
is False
, returns a loss per batch element instead and ignores size_average
. Default: True
'none'
| 'mean'
| 'sum'
. 'none'
: no reduction will be applied, 'mean'
: the sum of the output will be divided by the number of elements in the output, 'sum'
: the output will be summed. Note: size_average
and reduce
are in the process of being deprecated, and in the meantime, specifying either of those two args will override reduction
. Default: 'mean'
torch.nn.functional.cosine_embedding_loss(input1, input2, target, margin=0, size_average=None, reduce=None, reduction='mean') → Tensor
[source]
See CosineEmbeddingLoss
for details.
torch.nn.functional.cross_entropy(input, target, weight=None, size_average=None, ignore_index=-100, reduce=None, reduction='mean')
[source]
This criterion combines log_softmax
and nll_loss
in a single function.
See CrossEntropyLoss
for details.
C = number of classes
or $(N, C, H, W)$ in case of 2D Loss, or $(N, C, d_1, d_2, ..., d_K)$ where $K \geq 1$ in the case of K-dimensional loss.C
reduction
). By default, the losses are averaged over each loss element in the batch. Note that for some losses, there multiple elements per sample. If the field size_average
is set to False
, the losses are instead summed for each minibatch. Ignored when reduce is False
. Default: True
size_average
is True
, the loss is averaged over non-ignored targets. Default: -100reduction
). By default, the losses are averaged or summed over observations for each minibatch depending on size_average
. When reduce
is False
, returns a loss per batch element instead and ignores size_average
. Default: True
'none'
| 'mean'
| 'sum'
. 'none'
: no reduction will be applied, 'mean'
: the sum of the output will be divided by the number of elements in the output, 'sum'
: the output will be summed. Note: size_average
and reduce
are in the process of being deprecated, and in the meantime, specifying either of those two args will override reduction
. Default: 'mean'
Examples:
>>> input = torch.randn(3, 5, requires_grad=True) >>> target = torch.randint(5, (3,), dtype=torch.int64) >>> loss = F.cross_entropy(input, target) >>> loss.backward()
torch.nn.functional.ctc_loss(log_probs, targets, input_lengths, target_lengths, blank=0, reduction='mean', zero_infinity=False)
[source]
The Connectionist Temporal Classification loss.
See CTCLoss
for details.
Note
In some circumstances when using the CUDA backend with CuDNN, this operator may select a nondeterministic algorithm to increase performance. If this is undesirable, you can try to make the operation deterministic (potentially at a performance cost) by setting torch.backends.cudnn.deterministic =
True
. Please see the notes on Reproducibility for background.
Note
When using the CUDA backend, this operation may induce nondeterministic behaviour in its backward pass that is not easily switched off. Please see the notes on Reproducibility for background.
C = number of characters in alphabet including blank
, T = input length
, and N = batch size
. The logarithmized probabilities of the outputs (e.g. obtained with torch.nn.functional.log_softmax()
).(sum(target_lengths))
. Targets cannot be blank. In the second form, the targets are assumed to be concatenated.'none'
| 'mean'
| 'sum'
. 'none'
: no reduction will be applied, 'mean'
: the output losses will be divided by the target lengths and then the mean over the batch is taken, 'sum'
: the output will be summed. Default: 'mean'
False
Infinite losses mainly occur when the inputs are too short to be aligned to the targets.Example:
>>> log_probs = torch.randn(50, 16, 20).log_softmax(2).detach().requires_grad_() >>> targets = torch.randint(1, 20, (16, 30), dtype=torch.long) >>> input_lengths = torch.full((16,), 50, dtype=torch.long) >>> target_lengths = torch.randint(10,30,(16,), dtype=torch.long) >>> loss = F.ctc_loss(log_probs, targets, input_lengths, target_lengths) >>> loss.backward()
torch.nn.functional.hinge_embedding_loss(input, target, margin=1.0, size_average=None, reduce=None, reduction='mean') → Tensor
[source]
See HingeEmbeddingLoss
for details.
torch.nn.functional.kl_div(input, target, size_average=None, reduce=None, reduction='mean', log_target=False)
[source]
The Kullback-Leibler divergence Loss
See KLDivLoss
for details.
reduction
). By default, the losses are averaged over each loss element in the batch. Note that for some losses, there multiple elements per sample. If the field size_average
is set to False
, the losses are instead summed for each minibatch. Ignored when reduce is False
. Default: True
reduction
). By default, the losses are averaged or summed over observations for each minibatch depending on size_average
. When reduce
is False
, returns a loss per batch element instead and ignores size_average
. Default: True
'none'
| 'batchmean'
| 'sum'
| 'mean'
. 'none'
: no reduction will be applied 'batchmean'
: the sum of the output will be divided by the batchsize 'sum'
: the output will be summed 'mean'
: the output will be divided by the number of elements in the output Default: 'mean'
target
is passed in the log space. It is recommended to pass certain distributions (like softmax
) in the log space to avoid numerical issues caused by explicit log
. Default: False
Note
size_average
and reduce
are in the process of being deprecated, and in the meantime, specifying either of those two args will override reduction
.
Note
:attr:reduction
= 'mean'
doesn’t return the true kl divergence value, please use :attr:reduction
= 'batchmean'
which aligns with KL math definition. In the next major release, 'mean'
will be changed to be the same as ‘batchmean’.
torch.nn.functional.l1_loss(input, target, size_average=None, reduce=None, reduction='mean') → Tensor
[source]
Function that takes the mean element-wise absolute value difference.
See L1Loss
for details.
torch.nn.functional.mse_loss(input, target, size_average=None, reduce=None, reduction='mean') → Tensor
[source]
Measures the element-wise mean squared error.
See MSELoss
for details.
torch.nn.functional.margin_ranking_loss(input1, input2, target, margin=0, size_average=None, reduce=None, reduction='mean') → Tensor
[source]
See MarginRankingLoss
for details.
torch.nn.functional.multilabel_margin_loss(input, target, size_average=None, reduce=None, reduction='mean') → Tensor
[source]
See MultiLabelMarginLoss
for details.
torch.nn.functional.multilabel_soft_margin_loss(input, target, weight=None, size_average=None) → Tensor
[source]
See MultiLabelSoftMarginLoss
for details.
torch.nn.functional.multi_margin_loss(input, target, p=1, margin=1.0, weight=None, size_average=None, reduce=None, reduction='mean')
[source]
reduce=None, reduction=’mean’) -> Tensor
See MultiMarginLoss
for details.
torch.nn.functional.nll_loss(input, target, weight=None, size_average=None, ignore_index=-100, reduce=None, reduction='mean')
[source]
The negative log likelihood loss.
See NLLLoss
for details.
C = number of classes
or $(N, C, H, W)$ in case of 2D Loss, or $(N, C, d_1, d_2, ..., d_K)$ where $K \geq 1$ in the case of K-dimensional loss.C
reduction
). By default, the losses are averaged over each loss element in the batch. Note that for some losses, there multiple elements per sample. If the field size_average
is set to False
, the losses are instead summed for each minibatch. Ignored when reduce is False
. Default: True
size_average
is True
, the loss is averaged over non-ignored targets. Default: -100reduction
). By default, the losses are averaged or summed over observations for each minibatch depending on size_average
. When reduce
is False
, returns a loss per batch element instead and ignores size_average
. Default: True
'none'
| 'mean'
| 'sum'
. 'none'
: no reduction will be applied, 'mean'
: the sum of the output will be divided by the number of elements in the output, 'sum'
: the output will be summed. Note: size_average
and reduce
are in the process of being deprecated, and in the meantime, specifying either of those two args will override reduction
. Default: 'mean'
Example:
>>> # input is of size N x C = 3 x 5 >>> input = torch.randn(3, 5, requires_grad=True) >>> # each element in target has to have 0 <= value < C >>> target = torch.tensor([1, 0, 4]) >>> output = F.nll_loss(F.log_softmax(input), target) >>> output.backward()
torch.nn.functional.smooth_l1_loss(input, target, size_average=None, reduce=None, reduction='mean', beta=1.0)
[source]
Function that uses a squared term if the absolute element-wise error falls below beta and an L1 term otherwise.
See SmoothL1Loss
for details.
torch.nn.functional.soft_margin_loss(input, target, size_average=None, reduce=None, reduction='mean') → Tensor
[source]
See SoftMarginLoss
for details.
torch.nn.functional.triplet_margin_loss(anchor, positive, negative, margin=1.0, p=2, eps=1e-06, swap=False, size_average=None, reduce=None, reduction='mean')
[source]
See TripletMarginLoss
for details
torch.nn.functional.triplet_margin_with_distance_loss(anchor, positive, negative, *, distance_function=None, margin=1.0, swap=False, reduction='mean')
[source]
See TripletMarginWithDistanceLoss
for details.
torch.nn.functional.pixel_shuffle()
Rearranges elements in a tensor of shape $(*, C \times r^2, H, W)$ to a tensor of shape $(*, C, H \times r, W \times r)$ .
See PixelShuffle
for details.
Examples:
>>> input = torch.randn(1, 9, 4, 4) >>> output = torch.nn.functional.pixel_shuffle(input, 3) >>> print(output.size()) torch.Size([1, 1, 12, 12])
torch.nn.functional.pad(input, pad, mode='constant', value=0)
Pads tensor.
The padding size by which to pad some dimensions of input
are described starting from the last dimension and moving forward. $\left\lfloor\frac{\text{len(pad)}}{2}\right\rfloor$ dimensions of input
will be padded. For example, to pad only the last dimension of the input tensor, then pad
has the form $(\text{padding\_left}, \text{padding\_right})$ ; to pad the last 2 dimensions of the input tensor, then use $(\text{padding\_left}, \text{padding\_right},$ $\text{padding\_top}, \text{padding\_bottom})$ ; to pad the last 3 dimensions, use $(\text{padding\_left}, \text{padding\_right},$ $\text{padding\_top}, \text{padding\_bottom}$ $\text{padding\_front}, \text{padding\_back})$ .
See torch.nn.ConstantPad2d
, torch.nn.ReflectionPad2d
, and torch.nn.ReplicationPad2d
for concrete examples on how each of the padding modes works. Constant padding is implemented for arbitrary dimensions. Replicate padding is implemented for padding the last 3 dimensions of 5D input tensor, or the last 2 dimensions of 4D input tensor, or the last dimension of 3D input tensor. Reflect padding is only implemented for padding the last 2 dimensions of 4D input tensor, or the last dimension of 3D input tensor.
Note
When using the CUDA backend, this operation may induce nondeterministic behaviour in its backward pass that is not easily switched off. Please see the notes on Reproducibility for background.
Examples:
>>> t4d = torch.empty(3, 3, 4, 2) >>> p1d = (1, 1) # pad last dim by 1 on each side >>> out = F.pad(t4d, p1d, "constant", 0) # effectively zero padding >>> print(out.size()) torch.Size([3, 3, 4, 4]) >>> p2d = (1, 1, 2, 2) # pad last dim by (1, 1) and 2nd to last by (2, 2) >>> out = F.pad(t4d, p2d, "constant", 0) >>> print(out.size()) torch.Size([3, 3, 8, 4]) >>> t4d = torch.empty(3, 3, 4, 2) >>> p3d = (0, 1, 2, 1, 3, 3) # pad by (0, 1), (2, 1), and (3, 3) >>> out = F.pad(t4d, p3d, "constant", 0) >>> print(out.size()) torch.Size([3, 9, 7, 3])
torch.nn.functional.interpolate(input, size=None, scale_factor=None, mode='nearest', align_corners=None, recompute_scale_factor=None)
[source]
Down/up samples the input to either the given size
or the given scale_factor
The algorithm used for interpolation is determined by mode
.
Currently temporal, spatial and volumetric sampling are supported, i.e. expected inputs are 3-D, 4-D or 5-D in shape.
The input dimensions are interpreted in the form: mini-batch x channels x [optional depth] x [optional height] x width
.
The modes available for resizing are: nearest
, linear
(3D-only), bilinear
, bicubic
(4D-only), trilinear
(5D-only), area
'nearest'
| 'linear'
| 'bilinear'
| 'bicubic'
| 'trilinear'
| 'area'
. Default: 'nearest'
True
, the input and output tensors are aligned by the center points of their corner pixels, preserving the values at the corner pixels. If set to False
, the input and output tensors are aligned by the corner points of their corner pixels, and the interpolation uses edge value padding for out-of-boundary values, making this operation independent of input size when scale_factor
is kept the same. This only has an effect when mode
is 'linear'
, 'bilinear'
, 'bicubic'
or 'trilinear'
. Default: False
scale_factor
is passed as a parameter, it is used to compute the output_size
. If recompute_scale_factor
is `False
or not specified, the passed-in scale_factor
will be used in the interpolation computation. Otherwise, a new scale_factor
will be computed based on the output and input sizes for use in the interpolation computation (i.e. the computation will be identical to if the computed output_size
were passed-in explicitly). Note that when scale_factor
is floating-point, the recomputed scale_factor may differ from the one passed in due to rounding and precision issues.Note
With mode='bicubic'
, it’s possible to cause overshoot, in other words it can produce negative values or values greater than 255 for images. Explicitly call result.clamp(min=0, max=255)
if you want to reduce the overshoot when displaying the image.
Warning
With align_corners = True
, the linearly interpolating modes (linear
, bilinear
, and trilinear
) don’t proportionally align the output and input pixels, and thus the output values can depend on the input size. This was the default behavior for these modes up to version 0.3.1. Since then, the default behavior is align_corners = False
. See Upsample
for concrete examples on how this affects the outputs.
Warning
When scale_factor is specified, if recompute_scale_factor=True, scale_factor is used to compute the output_size which will then be used to infer new scales for the interpolation. The default behavior for recompute_scale_factor changed to False in 1.6.0, and scale_factor is used in the interpolation calculation.
Note
When using the CUDA backend, this operation may induce nondeterministic behaviour in its backward pass that is not easily switched off. Please see the notes on Reproducibility for background.
torch.nn.functional.upsample(input, size=None, scale_factor=None, mode='nearest', align_corners=None)
[source]
Upsamples the input to either the given size
or the given scale_factor
Warning
This function is deprecated in favor of torch.nn.functional.interpolate()
. This is equivalent with nn.functional.interpolate(...)
.
Note
When using the CUDA backend, this operation may induce nondeterministic behaviour in its backward pass that is not easily switched off. Please see the notes on Reproducibility for background.
The algorithm used for upsampling is determined by mode
.
Currently temporal, spatial and volumetric upsampling are supported, i.e. expected inputs are 3-D, 4-D or 5-D in shape.
The input dimensions are interpreted in the form: mini-batch x channels x [optional depth] x [optional height] x width
.
The modes available for upsampling are: nearest
, linear
(3D-only), bilinear
, bicubic
(4D-only), trilinear
(5D-only)
'nearest'
| 'linear'
| 'bilinear'
| 'bicubic'
| 'trilinear'
. Default: 'nearest'
True
, the input and output tensors are aligned by the center points of their corner pixels, preserving the values at the corner pixels. If set to False
, the input and output tensors are aligned by the corner points of their corner pixels, and the interpolation uses edge value padding for out-of-boundary values, making this operation independent of input size when scale_factor
is kept the same. This only has an effect when mode
is 'linear'
, 'bilinear'
, 'bicubic'
or 'trilinear'
. Default: False
Note
With mode='bicubic'
, it’s possible to cause overshoot, in other words it can produce negative values or values greater than 255 for images. Explicitly call result.clamp(min=0, max=255)
if you want to reduce the overshoot when displaying the image.
Warning
With align_corners = True
, the linearly interpolating modes (linear
, bilinear
, and trilinear
) don’t proportionally align the output and input pixels, and thus the output values can depend on the input size. This was the default behavior for these modes up to version 0.3.1. Since then, the default behavior is align_corners = False
. See Upsample
for concrete examples on how this affects the outputs.
torch.nn.functional.upsample_nearest(input, size=None, scale_factor=None)
[source]
Upsamples the input, using nearest neighbours’ pixel values.
Warning
This function is deprecated in favor of torch.nn.functional.interpolate()
. This is equivalent with nn.functional.interpolate(..., mode='nearest')
.
Currently spatial and volumetric upsampling are supported (i.e. expected inputs are 4 or 5 dimensional).
Note
When using the CUDA backend, this operation may induce nondeterministic behaviour in its backward pass that is not easily switched off. Please see the notes on Reproducibility for background.
torch.nn.functional.upsample_bilinear(input, size=None, scale_factor=None)
[source]
Upsamples the input, using bilinear upsampling.
Warning
This function is deprecated in favor of torch.nn.functional.interpolate()
. This is equivalent with nn.functional.interpolate(..., mode='bilinear', align_corners=True)
.
Expected inputs are spatial (4 dimensional). Use upsample_trilinear
fo volumetric (5 dimensional) inputs.
Note
When using the CUDA backend, this operation may induce nondeterministic behaviour in its backward pass that is not easily switched off. Please see the notes on Reproducibility for background.
torch.nn.functional.grid_sample(input, grid, mode='bilinear', padding_mode='zeros', align_corners=None)
[source]
Given an input
and a flow-field grid
, computes the output
using input
values and pixel locations from grid
.
Currently, only spatial (4-D) and volumetric (5-D) input
are supported.
In the spatial (4-D) case, for input
with shape $(N, C, H_\text{in}, W_\text{in})$ and grid
with shape $(N, H_\text{out}, W_\text{out}, 2)$ , the output will have shape $(N, C, H_\text{out}, W_\text{out})$ .
For each output location output[n, :, h, w]
, the size-2 vector grid[n, h, w]
specifies input
pixel locations x
and y
, which are used to interpolate the output value output[n, :, h, w]
. In the case of 5D inputs, grid[n, d, h, w]
specifies the x
, y
, z
pixel locations for interpolating output[n, :, d, h, w]
. mode
argument specifies nearest
or bilinear
interpolation method to sample the input pixels.
grid
specifies the sampling pixel locations normalized by the input
spatial dimensions. Therefore, it should have most values in the range of [-1, 1]
. For example, values x = -1, y = -1
is the left-top pixel of input
, and values x = 1, y = 1
is the right-bottom pixel of input
.
If grid
has values outside the range of [-1, 1]
, the corresponding outputs are handled as defined by padding_mode
. Options are
padding_mode="zeros"
: use 0
for out-of-bound grid locations,padding_mode="border"
: use border values for out-of-bound grid locations,padding_mode="reflection"
: use values at locations reflected by the border for out-of-bound grid locations. For location far away from the border, it will keep being reflected until becoming in bound, e.g., (normalized) pixel location x = -3.5
reflects by border -1
and becomes x' = 1.5
, then reflects by border 1
and becomes x'' = -0.5
.Note
This function is often used in conjunction with affine_grid()
to build Spatial Transformer Networks .
Note
When using the CUDA backend, this operation may induce nondeterministic behaviour in its backward pass that is not easily switched off. Please see the notes on Reproducibility for background.
Note
NaN values in grid
would be interpreted as -1
.
'bilinear'
| 'nearest'
. Default: 'bilinear'
Note: When mode='bilinear'
and the input is 5-D, the interpolation mode used internally will actually be trilinear. However, when the input is 4-D, the interpolation mode will legitimately be bilinear.'zeros'
| 'border'
| 'reflection'
. Default: 'zeros'
True
, the extrema (-1
and 1
) are considered as referring to the center points of the input’s corner pixels. If set to False
, they are instead considered as referring to the corner points of the input’s corner pixels, making the sampling more resolution agnostic. This option parallels the align_corners
option in interpolate()
, and so whichever option is used here should also be used there to resize the input image before grid sampling. Default: False
output Tensor
output (Tensor)
Warning
When align_corners = True
, the grid positions depend on the pixel size relative to the input image size, and so the locations sampled by grid_sample()
will differ for the same input given at different resolutions (that is, after being upsampled or downsampled). The default behavior up to version 1.2.0 was align_corners = True
. Since then, the default behavior has been changed to align_corners = False
, in order to bring it in line with the default for interpolate()
.
torch.nn.functional.affine_grid(theta, size, align_corners=None)
[source]
Generates a 2D or 3D flow field (sampling grid), given a batch of affine matrices theta
.
Note
This function is often used in conjunction with grid_sample()
to build Spatial Transformer Networks .
True
, consider -1
and 1
to refer to the centers of the corner pixels rather than the image corners. Refer to grid_sample()
for a more complete description. A grid generated by affine_grid()
should be passed to grid_sample()
with the same setting for this option. Default: False
output Tensor of size ($N \times H \times W \times 2$ )
output (Tensor)
Warning
When align_corners = True
, the grid positions depend on the pixel size relative to the input image size, and so the locations sampled by grid_sample()
will differ for the same input given at different resolutions (that is, after being upsampled or downsampled). The default behavior up to version 1.2.0 was align_corners = True
. Since then, the default behavior has been changed to align_corners = False
, in order to bring it in line with the default for interpolate()
.
Warning
When align_corners = True
, 2D affine transforms on 1D data and 3D affine transforms on 2D data (that is, when one of the spatial dimensions has unit size) are ill-defined, and not an intended use case. This is not a problem when align_corners = False
. Up to version 1.2.0, all grid points along a unit dimension were considered arbitrarily to be at -1
. From version 1.3.0, under align_corners = True
all grid points along a unit dimension are considered to be at `0
(the center of the input image).
torch.nn.parallel.data_parallel(module, inputs, device_ids=None, output_device=None, dim=0, module_kwargs=None)
[source]
Evaluates module(input) in parallel across the GPUs given in device_ids.
This is the functional version of the DataParallel module.
a Tensor containing the result of module(input) located on output_device
© 2019 Torch Contributors
Licensed under the 3-clause BSD License.
https://pytorch.org/docs/1.7.0/nn.functional.html