Skip to main content

Deep learning

This is where I write notes about code, applications, algorithms, everything related to deep learning


Word embeddings

Skip-gram models and negative sampling

Sigmoid, tanh, ReLU functions. What are they and when to use which?

Underfitting, Overfitting or Bias and Variance

Pytorch and Keras cheat sheets

Deep learning with Multi-GPUs





Comments

Post a Comment

Popular posts from this blog

Pytorch and Keras cheat sheets

Post a Comment
Read more

openNLP: getting started, code example

openNLP is an interesting tool for Natural Language Processing. Today, it took me a while to get started. So I want to write again that next time (or anyone) who wants to try it out, it will take less time for you. Here we go: 1. Download and build: This is the main website: http://opennlp.sourceforge.net/ You can either download the package there and build to the .jar file (where you have to set your $JAVA_HOME environment - see below). Or you can directly download the .jar file from this link . This step took me a while since I didn't know how to set my $JAVA_HOME, and didn't find out that there's already a .jar file to download. 2. Some code So now, you want to start with some code. Here is some sample code for doing Sentence Detection and Tokenization. Note that you can either download the models from the previous website or have the training dataset yourself. In this example, I used 2 models of openNLP (EnglishSD.bin.gz and EnglishTok.bin.gz). //This is ...
Post a Comment
Read more

Sigmoid, tanh, ReLU functions. What are they and when to use which?

If you are working on Deep Learning or Machine Learning in general, you have heard of these three functions quite frequently. We know that they can all be used as activation functions in neural networks. But what are these functions and why do people use for example ReLU in this part, sigmoid in another part and so on? Here is a friendly introduction to these functions and a brief explanation of when to use which. Sigmoid function Output from 0 to 1 Exponential computation (hence, slow) Is usually used for binary classification (when output is 0 or 1) Almost never used (e.g., tanh is a better option) Tanh function A rescaled logistic sigmoid function (center at 0) Exponential computation Works better than sigmoid ReLU function (Rectified Linear Unit) and its variants Faster to compute Often used as default for activation function in hidden layers ReLU is a simple model which gives 0 value to all W*x + b < 0. The importance is that it introduces t...
Post a Comment
Read more