Normalization
Major tasks of preprocessing are: Data cleaning filling missing values smoothing of noisy data identifying and removing outliers resolving inconsistencies Data Integration integrating data from multiple databases, data file, cubes Data transformation normalization aggregation Data reduction obtain a reduced representation of data but same results Data discretization part of data reduction but with particular importance, especially for numeric data Normalization : The goal of normalization is to make an entire set of values have a particular property. There are 3 different ways to perform normalization : min-max normalization X_std = (X - X.min(axis=0)) / (X.max(axis=0) - X.min(axis=0)) X_scaled = X_std * (max - min) + min here max, min is the new ranges z-score normalization z = (x - u) / s where u is the mean of the training samples, s is the standard deviation normalization by decimal scaling v_new = v/pow(10,j) ...