Weka

"Weka" stands for the Waikato Environment for Knowledge Analysis. (Also, the weka, pronounced to rhyme with Mecca, is a flightless bird with an inquisitive nature found only on the islands of New Zealand.) The system is written in Java, an objectoriented programming language that is widely available for all major computer platforms, and Weka has been tested under Linux, Windows, and Macintosh operating systems. Java allows us to provide a uniform interface to many different learning algorithms, along with methods for pre- and postprocessing and for evaluating the result of learning schemes on any given dataset. The interface is described in this chapter.

There are several different levels at which Weka can be used. First of all, it provides implementations of state-of-the-art learning algorithms that you can apply to your dataset from the command line. It also includes a variety of tools for transforming datasets, like the algorithms for discretization discussed in Chapter 7. You can preprocess a dataset, feed it into a learning scheme, and analyze the resulting classifier and its performance—all without writing any program code at all. As an example to get you started, we will explain how to transform a spreadsheet into a dataset with the right format for this process, and how to build a decision tree from it.

Learning how to build decision trees is just the beginning: there are many other algorithms to explore. The most important resource for navigating through the software is the online documentation, which has been automatically generated from the source code and concisely reflects its structure. We will explain how to use this documentation and identify Weka’s major building blocks, highlighting which parts contain supervised learning methods, which contain tools for data preprocessing, and which contain methods for other learning schemes. The online documentation is very helpful even if you do no more than process datasets from the command line, because it is the only complete list of available algorithms.

Weka is continually growing, and—being generated automatically from the source code—the online documentation is always up to date. Moreover, it becomes essential if you want to proceed to the next level and access the library from your own Java programs, or to write and test learning schemes of your own.

One way of using Weka is to apply a learning method to a dataset and analyze its output to extract information about the data. Another is to apply several learners and compare their performance in order to choose one for prediction. The learning methods are called classifiers. They all have the same command-line interface, and there is a set of generic command-line options—as well as some scheme-specific ones. The performance of all classifiers is measured by a common evaluation module. We explain the command-line options and show how to interpret the output of the evaluation procedure. We describe the output of decision and model trees. We include a list of the major learning schemes and their most important scheme-specific options. In addition, we show you how to test the capabilities of a particular learning scheme, and how to obtain a bias-variance decomposition of its performance on any given dataset.

Implementations of actual learning schemes are the most valuable resource that Weka provides. But tools for preprocessing the data, called filters, come a close second. Like classifiers, filters have a standardized command-line interface, and there is a basic set of command-line options that they all have in common. We will show how different filters can be used, list the filter algorithms, and describe their scheme-specific options.

The main focus of Weka is on classifier and filter algorithms. However, it also includes implementations of algorithms for learning association rules and for clustering data for which no class value is specified. We briefly discuss how to use these implementations, and point out their limitations.

GETTING STARTED
In most data mining applications, the machine learning component is just a small part of a far larger software system. If you intend to write a data mining application, you will want to access the programs in Weka from inside your own code. By doing so, you can solve the machine learning subproblem of your application with a minimum of additional programming. We show you how to do that by presenting an example of a simple data mining application in Java. This will enable you to become familiar with the basic data structures in Weka, representing instances, classifiers, and filters.

If you intend to become an expert in machine learning algorithms (or, indeed, if you already are one), you’ll probably want to implement your own algorithms without having to address such mundane details as reading the data from a file, implementing filtering algorithms, or providing code to evaluate the results. If so, we have good news for you: Weka already includes all this. In order to make full use of it, you must become acquainted with the basic data structures. To help you reach this point, we discuss these structures in more detail and explain example implementations of a classifier and a filter.