Prevent showing the edits icon?

If you wanted to do CSS, instead of display: none maybe visibility: hidden or opacity could work,

i.e. the pencil would still be there, but being not seen one would have to accidentally or intentionally click there.