What you are saying is that the rec button in the software is not reacting? The thing with questions is, the more information you give the better your fellow readers can help.
If the question is “how do I record” with no additional info like “I tried the record button, but it’s greyed out” or “I can’t find the record button” or … well, you get my drift, then we’ll start with the most obvious answer, won’t we?
For the recording function to work on DJ software you do have to use internal mixer mode, as otherwise the master output is put together in your controller and not in your software. And the software won’t know how to record the master output. I don’t know the S8 well enough, but it being Traktor gear, I am assuming that using internal Traktor mixing mode is the default setting.
Alternatively, you can always tap the master output from your controller, things like the Reloop Tape:
http://www.reloop.com/reloop-tape
and the new DJ Tech usb-stick recorder (see article Best Of The Rest at Frankfurt over on the blog) will allow you to record anything on any gear (so you could even use it when playing out on a regular CDJ/DJM club setup if you’d want.