When you comment, if you are not logged in to WordPress somewhere, you’ll see a new checkbox. Please tick the box “Save my name, email, and website in this browser for the next time I comment.”
This will set a cookie to remember you and will also show a moderation label (I moderate all comments) AND let you edit your comment for up to 5 minutes. If you don’t check the box, you can still comment but it won’t acknowledge you commented and you won’t be able to edit it.
PS: I souped up the comment form a bit and have added a link below the comment form with these tips.