Deep Learning Tricks

Efficient BackProp(Neural Networks: Tricks of the Trade, 2nd)

http://blog.csdn.net/zouxy09/article/details/45288129

Deep Learning for Vision: Tricks of the Trade(CVPR. Marc’Aurelio Ranzato)

http://bavm2013.splashthat.com/img/events/46439/assets/34a7.ranzato.pdf

Optimizing RNN performance(Silicon Valley AI Lab)

• intro: Optimize GEMM, parallel GPU, GRU and LSTM…

http://svail.github.io/

Must Know Tips/Tricks in Deep Neural Networks(NJU LAMDA, Xiu-Shen Wei)

• blog: http://lamda.nju.edu.cn/weixs/project/CNNTricks/CNNTricks.html

Training Tricks from Deeplearning4j

http://deeplearning4j.org/trainingtricks.html

Suggestions for DL from Llya Sutskeve

• intro: data, preprocessing, mini-batch, gradient normalization, learning rate, weight initialization, data augmentation, dropout and ensemble

http://yyue.blogspot.com/2015/01/a-brief-overview-of-deep-learning.html

Efficient Training Strategies for Deep Neural Network Language Models

• intro: batch-size, initial learning rate, network initialization

https://fb56552f-a-62cb3a1a-s-sites.googlegroups.com/site/deeplearningworkshopnips2014/71.pdf?attachauth=ANoY7cp_eDwTXPm6iWHdBRhlIsgPASEAwkW-exLSOsz467mge7zLCkBMWznOu_G90vGVtqNvXOusc4z6cC6hEnHk6YzHtuEr_kyU0fyme7asaECN0zvoNwDk5258CueoB6fY3WtLvbJzYok1xiIeWSFYtk5mKXCXFDMI6djwhjCX1xi0GEEv_x7uMQwTdQlDItZ3kgLnZ2RjctQmIXDCu58fS3Wby4vWX3CkhMIf_EpCXx7jDn_M2SM%3D&attredirects=0

Neural Networks Best Practice(Uber)

http://www.kentran.net/2013/04/neural-network-best-practices.html

How transferable are features in deep neural networks?(NIPS 2014)

http://papers.nips.cc/paper/5347-how-transferable-are-features-in-deep-neural-networks.pdf

Dark Knowledge from Hinton

• video: https://www.youtube.com/watch?v=EK61htlw8hY
• PPT: http://www.ttic.edu/dl/dark14.pdf
• notes: http://deepdish.io/2014/10/28/hintons-dark-knowledge/
• notes: http://fastml.com/geoff-hintons-dark-knowledge/

Stochastic Gradient Descent Tricks(Leon Bottou)

http://leon.bottou.org/publications/pdf/tricks-2012.pdf

Advice for applying Machine Learning

https://jmetzen.github.io/2015-01-29/ml_advice.html

How to Debug Learning Algorithm for Regression Model

http://vitalflux.com/machine-learning-debug-learning-algorithm-regression-model/

Large-scale L-BFGS using MapReduce(NIPS 2014)

http://papers.nips.cc/paper/5333-large-scale-l-bfgs-using-mapreduce.pdf

Selecting good features

Selecting good features – Part I: univariate selection:
http://blog.datadive.net/selecting-good-features-part-i-univariate-selection/

Selecting good features – Part II: linear models and regularization:
http://blog.datadive.net/selecting-good-features-part-ii-linear-models-and-regularization/

Selecting good features – Part III: random forests:
http://blog.datadive.net/selecting-good-features-part-iii-random-forests/

Selecting good features – Part IV: stability selection, RFE and everything side by side:
http://blog.datadive.net/selecting-good-features-part-iv-stability-selection-rfe-and-everything-side-by-side/

机器学习代码心得之​有监督学习的模块

http://www.weibo.com/p/1001603795687165852957

Stochastic Gradient Boosting: Choosing the Best Number of Iterations(Kaggle winner YANIR SEROUSSI)

http://yanirseroussi.com/2014/12/29/stochastic-gradient-boosting-choosing-the-best-number-of-iterations/

Large-Scale High-Precision Topic Modeling on Twitter(Twitter senior researcher. KDD 2014)

http://www.eeshyang.com/papers/KDD14Jubjub.pdf

H2O World - Top 10 Deep Learning Tips & Tricks - Arno Candel

http://www.slideshare.net/0xdata/h2o-world-top-10-deep-learning-tips-tricks-arno-candel