nabu.neuralnetworks.components.attention.WindowedAttention Class Reference

attention mechanism that is location aware More...

Inheritance diagram for nabu.neuralnetworks.components.attention.WindowedAttention:

Public Member Functions
def	__init__ (self, num_units, left_window_width, right_window_width, memory, memory_sequence_length=None, normalize=False, probability_fn=None, score_mask_value=float("-inf"), dtype=None, name='LocationAwareAttention')
	Construct the Attention mechanism. More...
def	initial_alignments (self, batch_size, dtype)
	get the initial alignments
def	__call__ (self, query, state)
	Score the query based on the keys and values. More...

Detailed Description

attention mechanism that is location aware

Constructor & Destructor Documentation

__init__()

def nabu.neuralnetworks.components.attention.WindowedAttention.__init__	(		self,
			num_units,
			left_window_width,
			right_window_width,
			memory,
			memory_sequence_length = None,
			normalize = False,
			probability_fn = None,
			score_mask_value = float("-inf"),
			dtype = None,
			name = 'LocationAwareAttention'
	)

Construct the Attention mechanism.

Parameters

num_units	The depth of the query mechanism.
window_width	the width of the attention window
memory	The memory to query; usually the output of an RNN encoder. This tensor should be shaped [batch_size, max_time, ...].
memory_sequence_length	Sequence lengths for the batch entries in memory. If provided, the memory tensor rows are masked with zeros for values past the respective sequence
lengths.
normalize	Python boolean. Whether to normalize the energy term.
probability_fn	(optional) A callable. Converts the score to probabilities. The default istf.nn.softmax}. Other options includetf.contrib.seq2seq.hardmax} and tf.contrib.sparsemax.sparsemax}. Its signature should be: probabilities = probability_fn(score).
score_mask_value	(optional): The mask value for score before passing into probability_fn. The default is -inf. Only used if memory_sequence_length is not None.
name	Name to use when creating ops.

Member Function Documentation

__call__()

def nabu.neuralnetworks.components.attention.WindowedAttention.__call__	(		self,
			query,
			state
	)

Score the query based on the keys and values.

Parameters

query	Tensor of dtype matching self.values and shape [batch_size, query_depth].
state	Tensor of dtype matching self.values and shape [batch_size, alignments_size] (alignments_size is memory's max_time).

Returns

alignments Tensor of dtype matching self.values and shape [batch_size, alignments_size] (alignments_size is memory's max_time).

---

The documentation for this class was generated from the following file:

• attention.py