Using Neural Networks to solve Regression Problems
The regression Neural Networks task builds a feed-forward neural network (NN) for regression. NN is a model that includes a certain number of elementary perceptrons organized in layers.
Each perceptron returns sigmoidal function (activation function) that depends on a weighted sum of its input. To retrieve the optimal weight vector for each perceptron a back-propagation algorithm is employed.
The output of the task is a model, consisting in a weight matrix ,that can be employed by an Apply Model task to perform the NN forecast on a set of examples.
Prerequisites
you must have created a flow;
the required datasets must have been imported into the flow;
the data used for the analysis must have been well prepared;
a unified model must have been created by merging all the datasets into the flow.
Additional tabs
The Monitor tab, where it is possible to view the temporal evolution of some quantities related to the network optimization. In particular, the behavior of error, gradient and misclassification is reported as a function of the number of iterations. These plots can be viewed during and after computation operations.
The Weights tab, where it is possible to view the weight matrix for each hidden layer. The layer to be displayed can be selected from the Chose Layer drop down list. Each row in the spreadsheet represents a neuron, while each column corresponds to an input attribute.
The Results tab, where statistics on the NN computation are displayed, such as the execution time.
Procedure
Drag the Neural Networks task onto the stage.
Connect a task, which contains the attributes from which you want to create the model, to the new task.
Double click the Neural Networks task. The left-hand pane displays a list of all the available attributes in the dataset, which can be ordered and searched as required.
Configure the options described in the table below.
Save and compute the task.
Neural Networks options | |
Parameter Name | Description |
|---|---|
Number of hidden layers | The number of hidden layers in the network. |
Neurons for each hidden layer | The number of neurons in each hidden layer. This option is enabled only if the value specified for the Number of hidden layers option is greater than zero. |
Learning rate | Indicate the coefficient of the gradient descent. For high values it is likely that the algorithm converges more quickly, but it may lack in precision. Alternatively, with low learning rates, the algorithm will converge more slowly, but with more accuracy. |
Maximum number of iterations | The (overall) maximum number of iterations for the gradient descent algorithm. |
Momentum term | Specify the momentum of the back-propagation algorithm. Momentum adds a fraction (between 0 and 1) of the previous weight update to prevent the system from converging to a local minimum. |
Number of trials | Select the number of trials, i.e. the repeated execution of back-propagation algorithm, with different starting conditions. |
Normalization for input variables | The type of normalization to use when treating ordered (discrete or continuous) variables. Possible methods are:
Every attribute can have its own value for this option, which can be set in the Data Manager task. These choices are preserved if Attribute is selected in the Normalization of input variables option; otherwise any selections made here overwrite previous selections made. |
Maximum number of iterations for any trial | The maximum number of iterations allowed for the gradient descent algorithm for each trial. |
Normalization of output variables | Select the method for normalizing output variables, in the same way as the Normalization for input variables. |
Use symmetric activation function | If selected, use an activation function in the range (-1,1) instead of (0,1). |
Aggregate data before processing | If selected, identical patterns are aggregated and considered as a single pattern during the training phase. |
Initialize random generator with seed | If selected, a seed, which defines the starting point in the sequence, is used during random generation operations. Consequently using the same seed each time will make each execution reproducible. Otherwise, each execution of the same task (with same options) may produce dissimilar results due to different random numbers being generated in some phases of the process. |
Append results | If selected, the results of this computation are appended to the dataset, otherwise they replace the results of previous computations. |
Input attributes | Drag and drop here the input attributes you want to use to build the network. Instead of manually dragging and dropping attributes, they can be defined via a filtered list. |
Output attributes | Drag and drop here the attributes you want to use to build the network. Instead of manually dragging and dropping attributes, they can be defined via a filtered list. |
Example
The following example uses the Adult dataset.
Description | Screenshot |
|---|---|
| 
|
The execution of the task can be monitored in the Monitor tab of the NN task. In these plots the behavior of the error as a function of the iteration is showed. Analogous plots can be visualized for covering and error by clicking on the corresponding tabs. |  |
Clicking on the Weights tab at the end of the execution, displays the weights obtained through the optimization procedure. The layer to be visualized can be selected in the Chose layer drop-down list: Each row corresponds to a neuron of that layer and contains the weights relative to each input attribute. The first column contains the bias, i.e. the constant term in the linear combination of inputs. |  |
Add an Apply Model task to the stage to apply the model to data The forecast ability of the generated model can be viewed by adding an Apply Model task to the NN task, and computing with default options. |  |
The forecast produced can be analyzed by right-clicking the Apply Model task and selecting Take a look. In the data table the following columns relative to the results of NN elaboration have been added:
|  |