Should outliers be removed or replaced?
If you drop outliers: Trim the data set, but replace outliers with the nearest “good” data, as opposed to truncating them completely. For example, if you thought all data points above the 95th percentile were outliers, you could set them to the 95th percentile value.
What can replace outliers?
Replacing Outliers with Median Values In this technique, we replace the extreme values with median values. It is advised to not use mean values as they are affected by outliers. The first line of code below prints the 50th percentile value, or the median, which comes out to be 140.
Is it justified to remove outliers?
However when an outlier has occurred from an error, the outlier is altering the data in false way, and can actually be beneficial to remove it. For example if a participant in a reaction time investigation, continually hits the button, even when there is no stimulus, their data is not going to be reliable or accurate. In a situation like this, I think it is justified to remove the outlier, as long as there has been made reference to it in the investigation report.
Is it dishonest to remove outliers?
In order to remove outliers, a method must be used to distinguish between the outliers and normal data. If you remove data that is not within the bounds of an outlier for your data set, you are unnecessarily transforming data, and therefore being dishonest with your results. It would therefore be dishonest to remove any outliers without declaring that you have done.
Should I remove outliers from my data?
Outliers can greatly influence our overall statistics and sometimes removing them from our data is the best option. However an outlier should only be removed if the result has not been gathered from the population that you were aiming to sample.
Why need to remove outlier?
There may be many truly valid reasons to remove data-points. These include outliers caused by measurement errors, incorrectly entered data-points or impossible values in real life. If you feel that any outlier are erroneous data points and you can validate this, then you should feel free to remove them.