Besides smooth woven fabric, you also have the option of wearing more casual knit shirts. These can be polos (in short or long sleeves), henleys, quarter zips and turtlenecks. All of these look fine ...
Extra credit: At nearly half off, we consider this babydoll short-sleeve top ... bestselling sweater dress is nearly 70% off right now! A hoodie design makes it the ideal casual outfit for outings ...
The Dorothy Perkins Leopard Keyhole Midi Dress is on offer for £4 off, now £34.20 down from £38. The piece is suitable for the colder climes thanks to its long sleeves and midi lengths.
The shirts you describe, which I would not particularly call dress shirts, are a smart-looking innovation that does not seem to have a name -- it is often described as "contrast fabric inside neckband ...