Skip to main content

Underfitting, Overfitting or Bias and Variance

In Machine Learning, we often hear the problems of underfitting, overfitting, or bias and variance.
What are they and how to "diagnose" the problem of your models?

Underfitting / high bias

  • Symptom: Your training error is high
  • Problem: Your model is not able to capture the underlying structures/relationships in your training data
  • Solution: Make your model more powerful (e.g., bigger nets, longer training time, more iterations)

Overfitting / high variance

  • Symptom: Errors on your development set are much higher than your training errors
  • Problem: Your model is "too fit" to the training data
  • Solution: Use more training data (or data augmentation - e.g., flip/rotate images to have more training samples), add regularization to the model (or some techniques like drop-out, early stopping, etc.)
Labels:

Comments

Post a Comment

Popular posts from this blog

Random variables

A random variable is a mapping from a sample space to real numbers $\Omega \rightarrow \mathrm{R}$ At a certain point in most probability courses, we don't see the sample space, but it's always there, lurking in the background. For example: Let $\Omega = \{(x,y); x^2 + y^2 \leq 1\}$ be the unit disc. Consider drawing a point "at random" from $\Omega$. Outcome: $\omega = (x,y)$. Examples of random variables: $X(\omega) = x$, $X(\omega) = y$, $Z(\omega) = x + y$
5 comments
Read more

SAXParser: too many exceptions for invalid XML character..

I'm working on my Similarity Search project, in which I have to implement the Tree Edit Distance and Traversal String Edit Distance. Trees are all represented in XML format and I'm using SAXParser to parse those XML files in java. I've used it a lot of times before but still, I don't quite like. So my first step is to create a valid XML database. However, "valid" to be parsed using SAXParser is complicated!! Here is what I get again and again: File Read Error: org.xml.sax.SAXParseException : The content of elements must consist of well-formed character data or markup. org.xml.sax.SAXParseException: The content of elements must consist of well-formed character data or markup. The reasons can be different, like: - Tags cannot contain number (e.g., is an invalid tag) - Tags cannot contain some symbols, like {, ., ?, etc. ("_" or "-" is fine) - Tags cannot be empty However, in my database, all of the tag are numbers.. To make it a valid XML fi...
2 comments
Read more