zhusuan.distributions

Distribution

class Distribution(dtype, is_continuous, is_reparameterized, use_path_derivative=False, group_ndims=0, device=device(type='cpu'), **kwargs)[source]

Bases: object

The Distribution class is the base class for various probabilistic distributions which support batch inputs, generating batches of samples and evaluate probabilities at batches of given values.

The typical input shape for a Distribution is like batch_shape + input_shape. where input_shape represents the shape of non-batch input parameter, batch_shape represents how many independent inputs are fed into the distribution.

Samples generated are of shape ([n_samples]+ )batch_shape + value_shape. The first additional axis is omitted only when passed n_samples is None (by default), in which case one sample is generated. value_shape is the non-batch value shape of the distribution. For a univariate distribution, its value_shape is [].

There are cases where a batch of random variables are grouped into a single event so that their probabilities should be computed together. This is achieved by setting group_ndims argument, which defaults to 0. The last group_ndims number of axes in batch_shape are grouped into a single event. For example, Normal(..., group_ndims=1) will set the last axis of its batch_shape to a single event, i.e., a multivariate Normal with identity covariance matrix.

When evaluating probabilities at given values, the given Tensor should be broadcastable to shape (... + )batch_shape + value_shape. The returned Tensor has shape (... + )batch_shape[:-group_ndims].

See also

For more details and examples, please refer to Basic Concepts in ZhuSuan.

For both, the parameter dtype represents type of samples. For discrete, can be set by user. For continuous, automatically determined from parameter types.

dtype must be among torch.int16, torch.int32, torch.int64, torch.float16, torch.float32 and torch.float64.

When two or more parameters are tensors and they have different type, TypeError will be raised.

Parameters

  • dtype – The value type of samples from the distribution.

  • is_continuous – Whether the distribution is continuous.

  • is_reparameterized – A bool. Whether the gradients of samples can and are allowed to propagate back into inputs, using the reparametrization trick from (Kingma, 2013).

  • use_path_derivative – A bool. Whether when taking the gradients of the log-probability to propagate them through the parameters of the distribution (False meaning you do propagate them). This is based on the paper “Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference”

  • group_ndims – A 0-D int32 Tensor representing the number of dimensions in batch_shape (counted from the end) that are grouped into a single event, so that their probabilities are calculated together. Default is 0, which means a single value is an event. See above for more detailed explanation.

property batch_shape

The shape showing how many independent inputs (which we call batches) are fed into the distribution. For batch inputs, the shape of a generated sample is batch_shape + value_shape.

property device

The device this distribution lies at.

Returns

torch.device

property dtype

The sample type of the distribution.

property is_reparameterized

Whether the gradients of samples can and are allowed to propagate back into inputs, using the reparametrization trick from (Kingma, 2013).

log_prob(given)[source]

Compute log probability density (mass) function at given value.

Parameters

given – A Var. The value at which to evaluate log probability density (mass) function. Must be able to broadcast to have a shape of (... + )batch_shape + value_shape.

Returns

A Var of shape (... + )batch_shape[:-group_ndims].

prob(given)[source]
sample(n_samples=None)[source]

Return samples from the distribution. When n_samples is None (by default), one sample of shape batch_shape + value_shape is generated. For a scalar n_samples, the returned Var has a new sample dimension with size n_samples inserted at axis=0, i.e., the shape of samples is [n_samples] + batch_shape + value_shape.

Parameters

n_samples – A 0-D int32 Tensor or None. How many independent samples to draw from the distribution.

Returns

A Var of samples.

Normal

class Normal(mean=0.0, std=None, logstd=None, dtype=None, is_continuous=True, is_reparameterized=True, group_ndims=0, device=device(type='cpu'), **kwargs)[source]

Bases: zhusuan.distributions.base.Distribution

The class of univariate Normal distribution. See Distribution for details.

Parameters

  • mean – A float Var. The mean of the Normal distribution. Should be broadcastable to match std or logstd.

  • std – A float Var. The standard deviation of the Normal distribution. Should be positive and broadcastable to match mean.

  • logstd – A float Var. The log standard deviation of the Normal distribution. Should be broadcastable to match mean.

  • group_ndims – A 0-D int32 Var representing the number of dimensions in batch_shape (counted from the end) that are grouped into a single event, so that their probabilities are calculated together. Default is 0, which means a single value is an event. See Distribution for more detailed explanation.

  • is_reparameterized – A Bool. If True, gradients on samples from this distribution are allowed to propagate into inputs, using the reparametrization trick from (Kingma, 2013).

  • use_path_derivative – A bool. Whether when taking the gradients of the log-probability to propagate them through the parameters of the distribution (False meaning you do propagate them). This is based on the paper “Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference”

property logstd

The log standard deviation of the Normal distribution.

property mean

The mean of the Normal distribution.

property std

The standard deviation of the Normal distribution.

Bernoulli

class Bernoulli(logits=None, probs=None, dtype=None, is_continuous=False, group_ndims=0, device=device(type='cpu'), **kwargs)[source]

Bases: zhusuan.distributions.base.Distribution

The class of univariate Bernoulli distribution. See Distribution for details.

Parameters

  • logits –

    A float Tensor. The log-odds of probabilities of being 1.

    \[\mathrm{logits} = \log \frac{p}{1 - p}\]

  • probs – A ‘float’ Tensor. The p param of bernoulli distribution

  • dtype – The value type of samples from the distribution. Can be int (torch.int16, torch.int32, torch.int64) or float (torch.float16, torch.float32, torch.float64). Default is int32.

  • group_ndims – A 0-D int32 Tensor representing the number of dimensions in batch_shape (counted from the end) that are grouped into a single event, so that their probabilities are calculated together. Default is 0, which means a single value is an event. See Distribution for more detailed explanation.

property logits
property probs

Beta

class Beta(alpha, beta, dtype=None, is_continuous=True, group_ndims=0, device=device(type='cpu'), **kwargs)[source]

Bases: zhusuan.distributions.base.Distribution

The class of univariate Beta distribution See Distribution for details.

Parameters

  • alpha – A ‘float’ Var. One of the two shape parameters of the Beta distribution.

  • beta – A ‘float’ Var. One of the two shape parameters of the Beta distribution.

property alpha

One of the two shape parameters of the Beta distribution.

property beta

One of the two shape parameters of the Beta distribution.

Exponential

class Exponential(rate, dtype=None, is_continuous=True, group_ndims=0, device=device(type='cpu'), **kwargs)[source]

Bases: zhusuan.distributions.base.Distribution

The class of univariate Exponential distribution See Distribution for details.

Parameters

rate – A ‘float’ Var. Rate parameter of the Exponential distribution.

property rate

Shape parameter of the Exponential distribution.

Gamma

class Gamma(alpha, beta, dtype=None, is_continuous=True, group_ndims=0, device=device(type='cpu'), **kwargs)[source]

Bases: zhusuan.distributions.base.Distribution

The class of univariate Gamma distribution See Distribution for details.

Parameters

  • alpha – A ‘float’ Var. Shape parameter of the Gamma distribution.

  • beta – A ‘float’ Var. Rate parameter of the Gamma distribution.

property alpha

Shape parameter of the Gamma distribution.

property beta

Rate parameter of the Gamma distribution.

Laplace

class Laplace(loc, scale, dtype=None, is_continuous=True, group_ndims=0, device=device(type='cpu'), **kwargs)[source]

Bases: zhusuan.distributions.base.Distribution

The class of univariate Laplace distribution See Distribution for details.

Parameters

  • loc – A ‘float’ Var. Mean of the Laplace distribution.

  • scale – A ‘float’ Var. Scale of the Laplace distribution.

property loc

Mean of the Laplace distribution.

property scale

Scale of the Laplace distribution.

Logistic

class Logistic(loc, scale, dtype=None, is_continuous=True, group_ndims=0, device=device(type='cpu'), **kwargs)[source]

Bases: zhusuan.distributions.base.Distribution

The class of univariate Logistic distribution, always using the reparametrization trick from (Kingma, 2013). See Distribution for details.

Parameters

  • loc – A ‘float’ Var. The location term acting on standard Logistic distribution.

  • scale – A ‘float’ Var. The scale term acting on standard Logistic distribution.

property loc
property scale

Poisson

class Poisson(rate, dtype=None, is_continuous=True, group_ndims=0, device=device(type='cpu'), **kwargs)[source]

Bases: zhusuan.distributions.base.Distribution

The class of univariate Poisson distribution See Distribution for details.

Parameters

rate – A ‘float’ Var. Rate parameter of the Poisson distribution.Must be positive.

property rate

Shape parameter of the Poisson distribution.

StudentT

class StudentT(df, loc=0.0, scale=1.0, dtype=None, is_continuous=True, group_ndims=0, device=device(type='cpu'), **kwargs)[source]

Bases: zhusuan.distributions.base.Distribution

The class of univariate StudentT distribution See Distribution for details.

Parameters

  • df – A ‘float’ Var. Degrees of freedom.

  • loc – A ‘float’ Var. Mean of the StudentT distribution.

  • scale – A ‘float’ Var. Scale of the StudentT distribution.

property df

Degrees of freedom.

property loc

Mean of the Laplace distribution.

property scale

Scale of the Laplace distribution.

Uniform

class Uniform(low, high, dtype=None, is_continuous=True, is_reparameterized=True, group_ndims=0, device=device(type='cpu'), **kwargs)[source]

Bases: zhusuan.distributions.base.Distribution

The class of univariate Uniform distribution See Distribution for details.

Parameters

  • low – A ‘float’ Var. Lower range (inclusive).

  • high – A ‘float’ Var. Upper range (exclusive).

property high

Upper range (exclusive) of the Uniform distribution.

property low

Lower range (inclusive) of the Uniform distribution.

FlowDistribution

class FlowDistribution(latents, transformation, flow_kwargs=None, dtype=torch.float32, group_ndims=0, device=device(type='cpu'), **kwargs)[source]

Bases: zhusuan.distributions.base.Distribution

A class for sample from Flow networks by provide the latent distribution and the flow network, when calling sample method, it returns the sample from flow network, when calling log_prob method it return the loss item of flow network.

Parameters

  • latents – An instance of Distribution class, as the prior（or the latent variable）of FlowDistrubution

  • transformation – A RevNet instance, the Flow net work built by user

  • flow_kwargs – additional info to be recode

  • dtype – data type

  • device – device of Distribution

utils

assert_same_dtype_in(tensors_with_name, dtypes=None)[source]

Whether all types of tensors in tensors_with_name are the same and in the allowed dtypes.

Parameters

  • tensors_with_name – A list of (tensor, tensor_name).

  • dtypes – A list of allowed dtypes. If None, then all dtypes are allowed.

Returns

The dtype of tensors.

assert_same_float_dtype(tensors_with_name)[source]

Whether all tensors in tensors_with_name have the same floating type.

Parameters

tensors_with_name – A list of (tensor, tensor_name).

Returns

The type of tensors.

assert_same_log_float_dtype(tensors_with_name)[source]

Whether all tensors in tensors_with_name have the same floating type, which also support log/exp operations.

Parameters

tensors_with_name – A list of (tensor, tensor_name).

Returns

The type of tensors.

check_broadcast(mean, std)[source]

check whether mean and std broadcast match