# Unsupervised 2 - Autoencoder

## Introduction

Supervised learning: fit(X, **Y**), train(X, **Y**), predict(X, **Y**)

Autoencoder: train(X, **X**), predict itself

* x is real value = squared error (regression)
* still use cross-entropy
* use sigmoid for a layer, since all output are \[0, 1] 
* share weights will be one option

```
# Not shared weight
Z = X * W_h + b_h
X_hat = Z * W_o + b_o

# shared weight
Z = X * W + b_h
X_hat = Z * Wtranspose + b_o
```

### PS:

sigmoid = binary classification (Sum may not be 1, and output is always in \[0, 1])

softmax = multi classification (Sum = 1)

## Regularization = way to generalization

1. append dataset, like rotate or shift image
2. add some noise(Gaussian): X + epsilon
3. set some X to 0