From the Plugins pane of the Options window, when you click the "Download and upgrade plugin to version..." button for a plugin, the "The plugin '[selected plugin]' will be upgraded to version [new version number] on next run of Picard." dialog box is displayed twice.

EDIT: If you then close and reopen the Options window and try selecting the "Download and upgrade plugin to version..." button again, the dialog box will be displayed an additional time.  Repeating the process seems to increment the number of times the dialog is displayed by two.