The obvious difference is having job wheels vs. having screens, but the S5 is really a replacement for the S2, not the S4. Despite having a 4-channel mixer, the S5 only has one external input, and does not support DVS (Traktor Scratch Pro). This design choice makes zero sense to me, but if you’re sure you can live with that limitation (plus lack of jog wheels) the S5 has slightly better build quality and can control remix decks in the absence of an F1. Unless you’re working completely by yourself, you will need a separate mixer to avoid shutting the sound off completely between DJs.
If possible, I’d go to a store that has floor models of both, and try them out. A few minutes of playing around will give you a feel for the difference. I own the S4 MK1 but I’ve played around on the S5 and S8. I wouldn’t trade my S4 for the S5 or the S8 as I like jog wheels, and I also like the ability to use the S4 as a central mixer with my friend’s CDJs on channels C and D.