I'd be fine if clothing and armor were separate, with the clothing being what's visible and the armor being what the stats come from.
But there is only so much your imagination can do. How does that tank top protect your arms? And again, why does Arasaka, a mega corp, give their employees armor...