The most appropriate place for deleting an image would be next to the buttons we show on hover for resizing images and adding alt text in the composer preview.
The hook to delete the image markup should be very similar to the 100%/75%/50% buttons. You can see here how the image scaling is done.