Model Selection

Do you know model selection? I heard it this afternoon. There were many related techniques. Originally, I believed it was about choosing a better model from several different models. For example, one of ME, one of SVM, other one is of Decision Trees. I did not know any other method except comparing the final accuracy and recall.

After discussion some friends, I knew it was about based on one model for choosing the best parameters. The over-fitting problem was well known. How to avoid it? We could use some criteria for measuring. For example, we could use Maximum Description Length and others.

It was different from selection of models of different mechanism. So I had one idea about selection. We could do a lot of selection, such as data selection, feature selection, models of different mechanism selection, model(parameter selection). There were so many selections, but how to select? It was a big problem. We should have a nice architecture to solve it.