Hi @james-smith,
There really is no established way to display bilingual content in Omeka without making some significant code changes in the theme.
One method for dealing with item metadata might be to duplicate each field that needs translation (just using the “add input” button would be my recommendation, but you can also create new custom fields) and have the first instance use English and the second use the Welsh equivalent. Show the English version by default (if that is the primary audience). Then you could add some JavaScript to the page, allowing the user to set an alternate language. When a user requests Welsh, you can use JavaScript to fetch that content using the Omeka REST API without reloading the page, replacing the English equivalent on the page. You could also use navigator.language to detect if the user has Welsh set as their default language.
For the Simple Pages plugin, you could enter both English and Welsh in their own HTML divs and have one hidden by default, using the same language selection method, and hiding/un-hiding each language as needed.
This level of customization may or may not be practical for you, but this is probably (more or less) how I would do it.
Sorry I don’t have a simpler solution. 