The hardest orchestral piece is less likely to be something that most people have heard of because such piece is not played often. Most times, the composer is trying to explore a music idea which can lead the composer to write a difficult piece. Some of these kind of piece are not well written. It would be difficult to give a definite answer to this question.
Mahler and Bartok symphonies are hard to play because they are very long. Mozart symphonies are hard to pull of. Lutoslawski's piece are also hard to play. The hardest orchestral piece might be a recent time piece that has only been played once and not known to many.