This has been available since IC614 (might have been earlier - I can't quite remember). The default is on, and it's controlled by these cdsenv:
ui.thumbnails enable boolean t
ui.thumbnails display boolean t
ui.thumbnails generate boolean t
If existing libraries don't have thumbnails, then in the Library Manager you can select a library and then do Edit->Update Thumbnails.