what effect changing a website from domain.com to domain.com.au or domain.net.au has on google rankings?

.au is a ccTLD which means it is specifically targeted to a locale, in this case australia.
Using locale-specific URLs (Country-specific domain) [support.google.com]

if your example.com was previously targeted to australia in the Country tab of the International Targeting report in GSC [support.google.com] it should essentially be equivalent.

domain Forward at the registrar level

the only thing you can control at the registrar level is specification of the nameserver.
if your registrar is also providing DNS and web hosting services, then that's how they are doing it - with a redirect on their web server.

Domain changes MAY cause changes in ranking. Some do well, others not so well. You will need to view results after at least 6 months to obtain enough data to review the results.

Meanwhile, you will need to keep both domains forever (so any 301s remain in effect) for such to work properly.