zhusuan.invertible

RevNet

class RevNet

Bases: torch.nn.modules.module.Module

An abc of reversible network，every subclass should implement both _forward and _inverse abstract method. return value of _forward and _inverse is like (y, log_det_J), in which y is the transformed tensor and log_det_J` is log-determinant of Jacobian.

forward(*inputs, reverse=False, **kwargs)

when using model.forward(x, reverse=False) process going with _forward(x), when using model.forward(x, reverse=True) process going with _inverse(x).

training: bool

RevSequential

class RevSequential(layers)

Bases: zhusuan.invertible.base.RevNet

the RevSequential provide a invertible transform which contain a list of instance of RevNet. when forward passing with reverse=False, the input x goes through every RevNet in the list also with reverse=False from begin to end , when forward passing with reverse=True, input x goes through every in the list also with reverse=True from end to begin.

Parameters

layers – a list of RevNet instance.

training: bool

Coupling

class Coupling(in_out_dim, mid_dim, hidden, mask_config)

Bases: zhusuan.invertible.base.RevNet

coupling layer class

Parameters

• in_out_dim – input/output dimensions.

• mid_dim – number of units in a hidden layer.

• hidden – number of hidden layers.

• mask_config – 1 if transform odd units, 0 if transform even units.

training: bool

MaskCoupling

class MaskCoupling(in_out_dim=- 1, mid_dim=- 1, hidden=- 1, mask=None, inner_nn=None)

Bases: zhusuan.invertible.base.RevNet

A coupling layer Identify if keep same or do transform by mask

Parameters

• in_out_dim – input/output dimensions.

• mid_dim – number of units in a hidden layer

• hidden – number of hidden layers

• mask – mask given by the user, often generated by function:~zhusuan.invertible.coupling.get_coupling_mask

training: bool

Scaling

class Scaling(dim)

Bases: zhusuan.invertible.base.RevNet

Initialize a (log-)scaling layer. when Forward pass, given class:x as input tensor, it returns (y, log_det_J) where y is transformed tensor by y=x*exp(log_scale) and log_det_J is log-determinant of Jacobian.

Parameters

dim – input/output dimensions.

training: bool

MaskedLinear

class MaskedLinear(input_size, n_outputs, mask, cond_label_size=None)

Bases: torch.nn.modules.linear.Linear

MADE building block layer

forward(x, cond_y=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

in_features: int

out_features: int

weight: torch.Tensor

MADE

class MADE(input_size, hidden_size, n_hidden, cond_label_size=None, input_order='sequential', input_degrees=None, activation='relu')

Bases: zhusuan.invertible.base.RevNet

MADE class

Parameters

• input_size – a scalar; dim of inputs

• hidden_size – a scalar; dim of hidden layers

• n_hidden – a scalar; number of hidden layers

• activation – a str; activation function to use

• input_order – a str or tensor; variable order for creating the autoregressive masks (sequential|random) or the order flipped from the previous layer in a stack of mades

• conditional – a bool; whether model is conditional

static create_mask(input_size, hidden_size, n_hidden, input_order='sequential', input_degrees=None)

Mask generator for MADE & MAF (see MADE paper sec 4:https://arxiv.org/abs/1502.03509)

Parameters

• input_size – dim of inputs

• hidden_size – dim of hidden layers

• n_hidden – number of hidden layers

• input_order – variable order for creating the autoregressive masks (sequential|random)

• input_degrees – degrees provide by user

Returns: List of masks

training: bool