Starting with Windows Vista, a new architecture for image decoding has been introduced and the built-in Windows Explorer is able to show thumbnails for image formats for which a suitable decoder has been installed. 3rd-party applications can also take advantage of the new software stack and benefit from installed codecs. Basically, all it takes to open and display DjVU files on Windows is a suitable decoder and a viewer with multi-page abilities.
A company called Caminova has developped the needed decoder, which can be downloaded here in 32 or 64-bit form. This software let Windows open the files, and create reduced-size views for display in Explorer (thumbnails).
FastPictureViewer Professional, an image viewer aimed at digital photographers, can take advantage of the DjVU codec for full-size multipage viewing of .DjVU files, with Direc3D hardware acceleration providing instant zooming and panning, plus advanced pre-loading and caching for near instantaneous navigation within the largest documents. The ability to view multi-page DjVU files is a nice side effect as this photo viewer was not originally designed for this type of viewing.
More information about DjVU can be found on djvu.org