Good black muck. Marmite/Vegemite is made from yeast extract, a byproduct of beer making. It has a strong salty flavor that some people have gone so far as to complain about in song. This is difficult to understand as the stuff contains large doses umami (that pleasant savory taste in everything from soy sauce to parmesan cheese). My theory is that "antimarmites" are just doing it wrong. This stuff is not to be glopped on or eaten off a spoon. A dab to flavor soup, a little spread on toast with cheese––so little does so much good. So what does it taste like? It is beefiness without the beef; it is all the good roasted bits.
So how to decide between the two? Strength of flavor seems to tip a little to Marmite, but Vegemite has the plus of spreadability. It all depends on what you are looking for in your tasty glop. I go with Vegemite simply because I live by the credo of No More Torn Toast.
Keep in mind that there is a third contender: a version made only for the Australasian market that is also confusingly called Marmite. It is made by a completely different company called Sanitarium. I prefer Kraft's Vegemite, but one gathers that Sanitarium's followers are equally committed.