This is an interesting piece from the Washington Post about ChatGPT witing false information: The WP calls it lies.
It's all new ground with any of this, but it's clear that fiction should not be taken as fact.

As largely unregulated artificial intelligence software such as ChatGPT, Microsoft’s Bing and Google’s Bard begins to be incorporated across the web, its propensity to generate potentially damaging falsehoods raises concerns about the spread of misinformation — and novel questions about who’s responsible when chatbots mislead.

[washingtonpost.com...]