最近俺们紫丁香BBS AI版上网友University撰写了“Machine Learning大家”系列帖子，特转载于此。University博闻强记，感谢！
发信人: university (侠之大者，学之小者), 信区: AI
标 题: Machine Learning 大家(1)：M.I.Jordan
发信站: BBS 哈工大紫丁香站 (Wed Aug 17 18:06:26 2005)
闲着无事，想写点一些我所了解的machine learning大家。由于学识浅薄，见识有限，并且仅局限于某些领域，一些在NLP及最近很热的生物信息领域活跃的学者我就浅陋无知，所以不对的地方大家仅当一笑。
在我的眼里，M Jordan无疑是武林中的泰山北斗。他师出了MIT， 现在在berkeley坐镇一方，在附近的两所名校（加stanford）中都可以说无出其右者，stanford的Daphne Koller虽然也声名遐迩，但是和Jordan比还是有一段距离。Jordan身兼stat和cs两个系的教授，从他身上可以看出Stat和ML的融合。
Jordan最先专注于mixtures of experts，并迅速奠定了自己的地位，我们的校友徐雷跟他做博后期间，也在这个方向上沾光不少。Jordan和他的弟子在很多方面作出了开创性的成果，如spectral clustering， Graphical model和nonparametric Bayesian。现在后两者在ML领域是非常炙手可热的两个方向，可以说很大程度上是Jordan的lab一手推动的。
更难能可贵的是，Jordan不仅自己武艺高强，并且揽钱有法，教育有方，手下门徒众多且很多人成了大器，隐然成为江湖大帮派。他的弟子中有10多人任教授，个人认为他现在的弟子中最出色的是stanford的Andrew Ng，不过由于资历原因，现在还是assistant professor，不过成为大教授指日可待；另外Tommi Jaakkola和David Blei也非常厉害，其中Tommi Jaakkola在mit任教而David Blei在cmu做博后。
并且由于形成了向心力，现在投奔Jordan的牛人非常多。如以前stanford的Ben Taskar， 数次获得NIPS最佳论文奖，把SVM的最大间隔方法和Markov network的structure结构结合起来，赫赫有名。还有一个博后是来自于toronto的Yee Whye Teh，非常不错，有幸跟他打过几次交道，人非常nice。另外还有一个博后居然在做生物信息方面的东西，看来jordan在这方面也捞了钱。
总的说来，我觉得Jordan现在做的主要还是graphical model和Bayesian learning，他去年写了一本关于graphical model的书，今年由mit press出版，应该是这个领域里程碑式的著作。3月份曾经有人答应给我一本打印本看看，因为Jordan不让他传播电子版，但后来好像没放在心上（可见美国人也不是很守信的），人不熟我也不好意思问着要，可以说是一大遗憾. 另外发现一个有趣的现象就是Jordan对hierarchical情有独钟，相当多的文章都是关于hierarchical的，所以能hierarchical大家赶快hierarchical，否则就让他给抢了。
发信人: university (侠之大者，学之小者), 信区: AI
标 题: Machine Learning大家（2）：D Koller
发信站: BBS 哈工大紫丁香站 (Thu Aug 18 15:25:58 2005)
最先知道D koller是因为她了一个大奖，不过不好意思的是这些奖的名字过于冗长并且拗口，所以我没记住。说起这个奖挺有意思的，80年代一个牛人指出机器只能独立学习而得到了这个奖，但是koller却因提出了Probabilistic Relational Models 而证明机器可以推理论知而又得到了这个奖，可见世事无绝对，科学有轮回。
D koller的Probabilistic Relational Models在nips和icml等各种牛会上活跃了相当长的一段时间，并且至少在实验室里证明了它在信息搜索上的价值，这也导致了她的很多学生进入了google。虽然进入google可能没有在牛校当faculty名声响亮，但要知道google的很多员工现在可都是百万富翁，在全美大肆买房买车的主。
Koller的研究主要都集中在probabilistic graphical model，如Bayesian网络，但这玩意我没有接触过，我只看过几篇他们的markov network的文章，但看了也就看了，一点想法都没有，这滩水有点深，不是我这种非科班出身的能趟的，并且感觉难以应用到我现在这个领域中。
Koller才从教10年，所以学生还没有涌现出太多的牛人，这也是她不能跟Jordan比拟的地方，并且由于在stanford的关系，很多学生直接去硅谷赚大钱去了，而没有在学术界开帮立派。其实大多数的professore都是希望自己的学生以后能当faculty，好在学术界形成更大的影响，但在stanford这可能太难以办到，因为金钱的诱惑实在太大了。不过Koller的一个学生我非常崇拜，叫Ben Taskar，就是我在（1）中所提到的Jordan的博后，是好几个牛会的最佳论文奖，他把SVM的最大间隔方法和Markov network结合起来，可以说是对structure data处理的一种标准工具，也把最大间隔方法带入了一个新的热潮，近几年很多牛会都有这样的workshop。 我最开始上Ben Taskar的在stanford的个人网页时，正赶上他刚毕业，他的顶上有这么一句话：流言变成了现实，我终于毕业了！ 可见Koller是很变态的，把自己的学生关得这么郁闷，这恐怕也是大多数女faculty的通病吧，并且估计还非常的push！
发信人: university (侠之大者，学之小者), 信区: AI
标 题: Machine learning 大家（3）:JD Lafferty
发信站: BBS 哈工大紫丁香站 (Sun Aug 21 16:37:16 2005)
大家都知道NIPS和ICML向来都是由大大小小的山头所割据，而John Lafferty无疑是里面相当高的一座高山，这一点可从他的publication list里的NIPS和ICML数目得到明证。虽然江湖传说计算机重镇CMU现在在走向衰落，但Lafferty却拥有越来越大的影响力，翻开AI兵器谱排名第一的journal of machine learning research的很多文章，我们都能发现author或者editor中赫然有Lafferty的名字。
Lafferty给人留下的最大的印象似乎是他2001年的conditional random fields，这篇文章后来被疯狂引用，广泛地应用在语言和图像处理，并随之出现了很多的变体，如Kumar的discriminative random fields等。虽然大家都知道discriminative learning好，但很久没有找到好的discriminative方法去处理这些具有丰富的contextual information的数据，直到Lafferty的出现。
而现在Lafferty做的东西好像很杂，semi－supervised learning， kernel learning，graphical models甚至manifold learning都有涉及，可能就是像武侠里一样只要学会了九阳神功，那么其它的武功就可以一窥而知其精髓了。这里面我最喜欢的是semi－supervised learning，因为随着要处理的数据越来越多，进行全部label过于困难，而完全unsupervised的方法又让人不太放心，在这种情况下semi－supervised learning就成了最好的折中方法。但可惜的是，个人认为学术界对如何用supervised信息控制未label的data似乎还没有一个比较清晰的认识，不过这也给了江湖后辈成名的可乘之机。到现在为止，我觉得cmu的semi－supervised是做得最好的，以前是KAMAL NIGAM做了开创性的工作，而现在Lafferty和他的弟子作出了很多总结和创新。
Lafferty的弟子好像不是很多，并且好像都不是很有名。不过今年毕业了一个中国人，Xiaojin Zhu，就是做semi－supervised的那个人，现在在wisc做assistant professor。他做了迄今为止最全面的Semi-supervised learning literature survey， 大家可以从他的个人主页中找到。这人看着很憨厚，估计是很好的陶瓷对象。另外我在（1）中所说的Jordan的牛弟子D Blei今年也投奔Lafferty做博后，就足见Lafferty的牛了。
2005年8月21日
2005年8月20日
Swimming & Research
I liked swimming now. I had tasted some nice feeling of it. This afternoon, Jianguo Du and Songnan Li came here for swimming by my invitation. This was my third time swimming in Beijing. At the second time, I only can breaststroke one time of twenty-five meters. At each end of 25 meters, I would inhale water. This afternoon, I tried more on breaststroke. Finally, I could swim 100 meters without inhaling Water.
I thought more about my progress of swimming. I found some link between swimming and doing research. Doing research, I believed the baseline system of your topic was very important. For construction the basic system, you should read many papers and design your architecture. Then you could make some simple modules one by one. Finally, you should link your modules, try once and evaluate it by some mechanism. The first done basic modules might be very poor. And your final system had very lower performance.
To swimming, you would be not able to at the beginning. Somebody will tell you how to swim and the motions. It was just like reading papers and knowing the whole architecture. Then you would have your own understanding about swimming. Then you would try to practice each motion. It was just like making each module. Affirmatively, you could not do best one each motions. You should try them one by one. In swimming, every fresh guy was excited. He would like to try the whole process at any time. So at first, he would fail once and once again. Only when he had done all the modules with better performance, he might success once. It was just like the baseline system of your research work. Then he tried the process by all means. Until now, he could understand the importance of each motion and try to upgrade them. During the process, his evaluation standard would change to be finer and finer. It's just like the first research experience on the baseline system. Then after fine feeling of each module during the running, each module would be improved. Finally the baseline system would become the best system. But this was the first method of swimming, based on the experience of the first baseline system, he would try other method. And after similar process, he would master more and more methods. And finally, he would link all the methods and found some commons and put forward some newly methods.
Yeah. Just like swimming, you must practice instead of keep standing on banks.
I thought more about my progress of swimming. I found some link between swimming and doing research. Doing research, I believed the baseline system of your topic was very important. For construction the basic system, you should read many papers and design your architecture. Then you could make some simple modules one by one. Finally, you should link your modules, try once and evaluate it by some mechanism. The first done basic modules might be very poor. And your final system had very lower performance.
To swimming, you would be not able to at the beginning. Somebody will tell you how to swim and the motions. It was just like reading papers and knowing the whole architecture. Then you would have your own understanding about swimming. Then you would try to practice each motion. It was just like making each module. Affirmatively, you could not do best one each motions. You should try them one by one. In swimming, every fresh guy was excited. He would like to try the whole process at any time. So at first, he would fail once and once again. Only when he had done all the modules with better performance, he might success once. It was just like the baseline system of your research work. Then he tried the process by all means. Until now, he could understand the importance of each motion and try to upgrade them. During the process, his evaluation standard would change to be finer and finer. It's just like the first research experience on the baseline system. Then after fine feeling of each module during the running, each module would be improved. Finally the baseline system would become the best system. But this was the first method of swimming, based on the experience of the first baseline system, he would try other method. And after similar process, he would master more and more methods. And finally, he would link all the methods and found some commons and put forward some newly methods.
Yeah. Just like swimming, you must practice instead of keep standing on banks.
2005年8月19日
Farewell to Chengjie Sun
Chengjie Sun will go back to Harbin for his visa to Korea. We three had a dinner on Caozhou Restaurant near Sigma Building.
2005年8月18日
Begin to try OneNote
It's a nice software just like the combination of MindManager and Tadalist. I used it to edit our project discussion result yesterday. It was convenient and feasible. I liked it and would try it more for my research and project.
2005年8月17日
Knowing what you will do is the first!
This is Aug. 17th. I had come here three months. It was half of our study in MSRA. During the past three months, I had learned a lot, knew many friends from different universities, listened many research talks with high-quality, did survey on my research topics, organized ten ping pong activities, and began to like swimming.
This afternoon, our project group had the routine meeting. We discussed only my recent tasks report. After my presentation, our mentor described my task scene in detail. Until now, we four VSs knew our tasks in detail again. As we had done much work on our project, our goals and tasks were more and more clearly.
Yeah! I believed that knowing what you will do was the first. Our four tasks were all novel and interesting. We all would concentrate on them. I believed we could do the research at the state of the art.
This was the weekly ping pong day. There were about ten accepted my invitation. We all could have a nice sport evening!
This afternoon, our project group had the routine meeting. We discussed only my recent tasks report. After my presentation, our mentor described my task scene in detail. Until now, we four VSs knew our tasks in detail again. As we had done much work on our project, our goals and tasks were more and more clearly.
Yeah! I believed that knowing what you will do was the first. Our four tasks were all novel and interesting. We all would concentrate on them. I believed we could do the research at the state of the art.
This was the weekly ping pong day. There were about ten accepted my invitation. We all could have a nice sport evening!
2005年8月16日
Do you master your tools?
Sharpening your axe will not delay your job of cutting wood. To our everyday using office software, such as word, outlook, excel, powerpoint, I believed nobody had mastered their total functions.
This evening, I read many training materials on office tools. They were about Outlook, Project and Onenote. During the reading, I found that Miscrosoft Sharepoint was a nice tool for team work. Onenote could be used as To Do List.
Why you not read them? Please visit: http://office.microsoft.com/zh-cn/default.aspx
This evening, I read many training materials on office tools. They were about Outlook, Project and Onenote. During the reading, I found that Miscrosoft Sharepoint was a nice tool for team work. Onenote could be used as To Do List.
Why you not read them? Please visit: http://office.microsoft.com/zh-cn/default.aspx
2005年8月15日
How to found an exciting and effective research organization?
To the title, it was a tremendous problem to the organizer. I had the experience on organizing the machine learning group of our university. I had encountered so many difficulties. And after three times colloquia, we had nearly no presentation volunteers and sharing topics.
I believed that now I knew more about how to solve this problem. Or I knew the important issues on this topic. In the second chapter of Complexity, there was a main subject about the motivation and the beginning of Santa Fe Institute. I had known nothing about this institute before reading this chapter. The goal of it was setting up a place for cross-subject scientists rap off. The first organizer George Cowan had the original idea on building Grand Unified Theories(GUT). It was based on complexity theory. During their first two colloquia, they had got more universal resonance. Then with the development and influence, there were more and more scientists who were interested in GUT on complexity. There was one thing important for this institute. That was so many Nobelists were its members. During the process, I realized that the most important thing for organizing such organization was how to pilot and mobilize the interesting of all.
During this chapter, there was one viewpoint which was more and more clearly. It was many research topics owned something same at some degree. This happened between many areas, such as physics, biology, biologist and economics. I thought, in chapter two, it was the complexity.
I considered that I was more like reading this book. It was good enough!
I believed that now I knew more about how to solve this problem. Or I knew the important issues on this topic. In the second chapter of Complexity, there was a main subject about the motivation and the beginning of Santa Fe Institute. I had known nothing about this institute before reading this chapter. The goal of it was setting up a place for cross-subject scientists rap off. The first organizer George Cowan had the original idea on building Grand Unified Theories(GUT). It was based on complexity theory. During their first two colloquia, they had got more universal resonance. Then with the development and influence, there were more and more scientists who were interested in GUT on complexity. There was one thing important for this institute. That was so many Nobelists were its members. During the process, I realized that the most important thing for organizing such organization was how to pilot and mobilize the interesting of all.
During this chapter, there was one viewpoint which was more and more clearly. It was many research topics owned something same at some degree. This happened between many areas, such as physics, biology, biologist and economics. I thought, in chapter two, it was the complexity.
I considered that I was more like reading this book. It was good enough!
订阅：
博文 (Atom)