Add below code snippet to your init.el:

(defun file-notify-rm-all-watches ()
  "Remove all existing file notification watches from Emacs."
  (interactive)
  (maphash
   (lambda (key _value)
     (file-notify-rm-watch key))
   file-notify-descriptors))

Then just execute M-x file-notify-rm-all-watches when you meet this problem next time.