Hi Irfan,
The ADC values do have a 2mV-3mV variation wrt fixed voltages. Yes by designing a good LPF you can reduce this variations. Averaging 10 ADC values would also be an option for accurate value but this increases the computation time and utilizes unnecessary cpu cycles. All ADC have same kind of variations please go through the ADCResults registers for details. Also please go through the ADC documentation "SPRU716D".
It is recommended that when some ADCs are ununsed , ground them.
Regards,
Gautam