First of all, be very very sure that your IP is not associated with the email in any way. To do this, obtain the full source of one of these spam emails and check that your IP doesn't occupy any of the lines. Also check your host's mailserver IP is not there as (in some rare cases) your mail can be sent with only the host's mailserver IP and not your own.

If your IP is there then you've got a serious problem with your server. Find a professional or contact your host/datacenter and get it fixed ASAP.

Assuming now that the spam did not originate through a hacked server or compromised software on your server, you can now investigate SPF. This will help you out with many of the major email services. It's only a small line to add to your DNS server setup, and it will result in any email sent from your domain name but not from your IP being marked as fraudulent spam and being undelivered.

<edit>
Just saw your edit...

I should have mentionned that I do not get any bounce
but am receiving emails looking like sent by myself

There are a lot of these emails going around. What better to attract your attention than your own email address? You expect to see your own email address on an email, just slightly higher up in the 'to' line :)

On the plus side - assuming you don't email yourself normally - email 'from' your address to your address is easy to block with a rule.
</edit>

Thanks
I am on a dedicated server and their IP is not there.
I really need (thanks to remind me about) to go the SPF route.

When SPF will be set at server level

do I have to do anything at email client level
in order to benefit from it?

also I think I may still use my old email address
is that correct?

No, nothing needs to be done at the individual client level, it is done on a per domain basis.

You can still use the same email address.

Be careful - I once broke the DNS by adding SPF incorrectly....