If a data set has an upper limit, then test on hardware with the maximum data set. It is very easy not to notice that some operation is very slow, or causes problems, if it is never tested on hardware at its limits. Always be sure to load up test scenarios with large data sets.