Unfortunately this is not an option in HTML, and each browser handles it differently, so it requires a few different approaches. See this post - https://www.joomlacontenteditor.net/support/forum/91927#p91932 And also, try adding the following to your template stylehseet:

audio::-internal-media-controls-download-button {
    display:none;
}

audio::-webkit-media-controls-enclosure {
    overflow:hidden;
}

audio::-webkit-media-controls-panel {
    width: calc(100% + 30px); /* Adjust as needed */
}