User:Francis Tyers/Perceptron

A perceptron is a classifier that

The classifier consists of:

• An input vector, $x$
• A weight vector, $w$
• Threshold, $\theta$

The input vector is made up of binary features, such as:

$h_{\mathrm {estacio'} }(t,c)={\begin{cases}1&{\text{if }}t~=~season~\mathrm {and} ~sec~\mathrm {follows} ~estacio'~\\0&{\text{otherwise}}\end{cases}}$

Training algorithm

The objective of the training algorithm is to find the most adequate set of weights, $w$ and a threshold $\theta$.

def decision(input, weights, threshold): #{


#}

# Initialise weights and threshold.
weights = [0.0, 0.0];
threshold = 0.0;
errors = 0;

while True: #{

        
        # If there are no errors, training has converged.
        if errors == 0: #{
                break;
        #}
#}

Example

Here is a worked example of a perceptron applied to the task of lexical selection. Lexical selection is the task of choosing a target translation $t*$ for a given source word $s$ in a context $c$ out of a set of possible translations $T$. A perceptron makes a classification decision for a single class, so we need to train a separate perceptron for each possible target word selection.

In the example,

• $s$ = estació
• $T$ = {season, station}
• $t*$ = season

Features

The features we will be working with are ngram contexts around the "problem word". These can be extracted from the word alignments calculated from a parallel corpus.

Catalan	English
Durant l' estació seca les pluges són escasses.	During the dry season it rains infrequently.
L' estiu és una estació de l' any.	Summer of one of the seasons of the year.
Barcelona-Sants és una estació de tren a Barcelona.	Barcelona-Sants is a train station in Barcelona.
...

Training data

s = estació, t* = season

$c_{i}$
_ sec	1
_ de el any	1
_ de tren	0
_ de el línia	0
_ humit	1
_ plujós	1
un _ a	0

Feature vector

This is the above training data expressed as an input vector $x_{j}$ to the perceptron.

_ sec	_ de el any	_ de tren	_ de la línia	_ humit	_ plujós	un _ a		$d$
1	0	0	0	0	0	0		1
0	1	0	0	0	0	0		1
0	0	1	0	0	0	0		0
0	0	0	1	0	0	0		0
0	0	0	0	1	0	0		1
0	0	0	0	0	1	0		1
0	0	0	0	0	0	1		0

Weight vector

$w_{0}$	$w_{1}$	$w_{2}$	$w_{3}$	$w_{4}$	$w_{5}$	$w_{6}$
_ sec	_ de el any	_ de tren	_ de la línia	_ humit	_ plujós	un _ a
0.0	0.0	0.0	0.0	0.0	0.0	0.0