I've just bought the new Pico, and one of these:

https://shop.pimoroni.com/products/pico-audio-pack

Even the device name is calling out for a PiCorePlayer port.

Not sure how long it will take to get here in France post Brexit, but could be a nice cheap player.