Don't be nervous! I'm one of the CAs who'll be at the assessment centres tomorrow. They're really not stressful. Last year, we played a few ice-breakers at the beginning, and you go around in a small group with other applicants for the day, so you aren't alone!
I wore khakis, a dress shirt, and a casual blazer (can't remember if I wore a tie or not..
But I don't think it would stand out in any negative way to be a bit more dressy than others). When it comes down to it, it's about the kind of person you are. The CAs weren't told to assess based on clothing.