Incorrect glyphs selected for angle brackets #18740

Bug report

Bug summary

mathtext sometimes selects incorrect glyphs to display angle brackets.

Code for reproduction

from pylab import *

figtext(.2, .7,
        r"$\left<?\right>"
        r"\left\langle?\right\rangle$") \
    .set_math_fontfamily("custom")
figtext(.2, .3,
        r"$\left<\frac{\sum_{0}^{1}}{\sum_{0}^{1}}\right>"
        r"\left\langle\frac{\sum_{0}^{1}}{\sum_{0}^{1}}\right\rangle$") \
    .set_math_fontfamily("cm")
show()

Actual outcome

The glyphs for the small \left<\right> when using "custom" mathtext.fontset and the large \left\langle\right\rangle when using "cm" mathtext.fontset are wrong.

Expected outcome

Correct glyphs.

I have some ideas on how to fix this, but would rather wait for mathtext to go fully private (i.e. 3.6) instead of jumping through hoops and hoops of backcompat layers... (but if anyone else wants to have a go at it before then, please do)

Matplotlib version

• Operating system: fedora
• Matplotlib version: head
• Matplotlib backend (print(matplotlib.get_backend())): agg
• Python version: 38
• Jupyter version (if applicable):
• Other libraries: