http://ufldl.stanford.edu/tutorial/

https://www.willamette.edu/~gorr/classes/cs449/momrate.html

http://sebastianruder.com/optimizing-gradient-descent/index.html#visualizationofalgorithms

In the neural network terminology:

one epoch = one forward pass and one backward pass of all the training examples

batch size = the number of training examples in one forward/backward pass. The higher the batch size, the more memory space you'll need.

number of iterations = number of passes, each pass using [batch size] number of examples. To be clear, one pass = one forward pass + one backward pass (we do not count the forward pass and backward pass as two different passes).

https://github.com/tensorflow/skflow/issues/138

Example: if you have 1000 training examples, and your batch size is 500, then it will take 2 iterations to complete 1 epoch.

shareimprove this answer

http://colah.github.io/posts/2014-07-NLP-RNNs-Representations/

in tensor flow: if there are n samples and the batch size is b, total steps is: s, then number of epochs will be s*b/n

```
STEPS = 5000
BATCH_SIZE = 10
periods = 10
steps_per_period = STEPS / periods
for period in range (0, periods):
..
```

https://www.quora.com/What-is-the-best-way-to-understand-the-terms-precision-and-recall

https://www.quora.com/Machine-Learning-How-does-grid-search-work

http://blog.algorithmia.com/introduction-machine-learning-developers/

https://news.ycombinator.com/item?id=12924020