JCE Captions were created before the <figure> tag existed, or was part of the standard HTML specification, so it was a way to create inline, styled captions.
The <figure> and <figcaption> tags are the "official" way to markup and caption images, with some caveats:
The <figure> tag is usually styled and displayed as a block element, which means it is on its own line, ie: you can't have something next to it like text, unless you are using Columns or some other sectioning markup.
By default the <figure> tag is styled with margins all around it, so you usually have to override this with a margin:0 style
The <figure> and <figcaption> tags are usually unstyled - without borders, colours etc. - so you will need to add any special styling yourself.
The following css seems to work well for generic <figure> and <figcaption> tags, which you can add to for additional styling: