“Is your optimization function correct?” Revisited

I think I have a cleaner way to explain Diagnosis Tip #2 in my post on diagnosing problems with your machine learning algorithm. Many machine learning solutions boil down to defining a model that is specified by some parameter $latex \theta$ and then creating an optimization function or error function $latex J( \theta )$ that […]

Cheat Sheet: Properties of Probability Distributions

Here is a probability distribution cheat sheet that I like to keep around for reference. This focuses on the “big picture” properties of some well known PDFs. The goal is to collect some properties that can help me decide when it’s appropriate to use a particular distribution. Beta Distribution Used in task duration modeling (E.g.. […]

Evidence approximation in linear regression: A method that produces “automatically regularized” solutions.

Summary In this post, I look at a Bayesian treatment of the linear regression problem. Making use of basis functions allows you to model non-linear patterns in data, however taking this route usually requires that you regularize your solution. To find the best regularization parameter often requires cross validation, but by looking at a framework […]

Some time series analysis resources

I have been looking into time series analysis topics recently. Here’s a list of resources that I have found useful so far. I’ll update this if I find more stuff: * Free PDF Text: A First Course on Time Series Analysis –  From the University of Wurzburg. * http://www.statsoft.com/textbook/time-series-analysis/ – This is actually a website […]