Does TOS/Privacy policy affect your SEO ranking anyhow?

not in itself but you could have links from them to other pages

Is it a good idea to have those documents on a website?

yes to protect you / your business