I’ve got it too. Having a subcategory named ‘edit’ turns the category link into an edit button.
As a workaround, to delete the glitching edit category I selected it from the dropdown rather than clicking a category link, and that will let you back into its settings to delete.
Nope, that did not work consistently.