Skip to main content

Underfitting, Overfitting or Bias and Variance

In Machine Learning, we often hear the problems of underfitting, overfitting, or bias and variance.
What are they and how to "diagnose" the problem of your models?

Underfitting / high bias

  • Symptom: Your training error is high
  • Problem: Your model is not able to capture the underlying structures/relationships in your training data
  • Solution: Make your model more powerful (e.g., bigger nets, longer training time, more iterations)

Overfitting / high variance

  • Symptom: Errors on your development set are much higher than your training errors
  • Problem: Your model is "too fit" to the training data
  • Solution: Use more training data (or data augmentation - e.g., flip/rotate images to have more training samples), add regularization to the model (or some techniques like drop-out, early stopping, etc.)
Labels:

Comments

Post a Comment

Popular posts from this blog

Quick text files merging, data preparation

It's very often that in natural language processing, you will have to re-format your data to take as inputs to different systems. In this case, these simple linux commands will help you do it much quicker without having to write a script. 1. Merging two files to one file with two column Input f1 looks like this: 1 2 3 4 Input f2 looks like this: a b c d Output f3 will look like this: 1  a 2  b 3  c 4  d Command: paste f1 f2 > f3  The delimiter by default is a tab. You can also define it (for example, separated by a comma) as follows: paste -d ',' f1 f2 > f3 2.  Create a line number to each line of a text file Assume that you want to create an index to each line in a text file, i.e. inserting a line number and then a tab before the content of each line: Input f1: a b c d Output f2: 1  a 2  b 3  c 4  d Command: nl f1 > f2 3. Joining two files with a common field Input f1: 1   aaa...
1 comment
Read more

SAXParser: too many exceptions for invalid XML character..

I'm working on my Similarity Search project, in which I have to implement the Tree Edit Distance and Traversal String Edit Distance. Trees are all represented in XML format and I'm using SAXParser to parse those XML files in java. I've used it a lot of times before but still, I don't quite like. So my first step is to create a valid XML database. However, "valid" to be parsed using SAXParser is complicated!! Here is what I get again and again: File Read Error: org.xml.sax.SAXParseException : The content of elements must consist of well-formed character data or markup. org.xml.sax.SAXParseException: The content of elements must consist of well-formed character data or markup. The reasons can be different, like: - Tags cannot contain number (e.g., is an invalid tag) - Tags cannot contain some symbols, like {, ., ?, etc. ("_" or "-" is fine) - Tags cannot be empty However, in my database, all of the tag are numbers.. To make it a valid XML fi...
2 comments
Read more