Great approach. that what I do in some cases even to use it as an OCR. you can add in your prompt "Describe the image in details". it will get closer to th real image. I've crafted this story for that case using DALL-E:

https://medium.com/ziadlit/secrets-to-design-a-free-3d-image-in-seconds-for-your-brand-using-dall-e-3-74ad01409190