What exactly are .thumbdata files and what is their purpose?
Actually, nobody knows exactly, well except for the devs, but they didn't share their info. There are just hints and deductions, but there are no exact specifications for the THUMBDATA format atm.
As was said before, *.thumbdata files are hidden by default, so unless you have the option for showing hidden files enabled, (Menu ► Settings ► Show Hidden Files) you might not even be aware of them and that they are eating up your free space. You can find them in /DCIM/.thumbnails folder on your SD card or internal storage, depending on where your images are saved by default.
Considering *.thumbdata files are generated in the DCIM folder where the digital camera saves photos, it is likely that the *.thumbdata files are generated by apps related to the digital camera, image preview or editing. From what we could gather, the general consensus is that these files are created by any of the media gallery apps.
THUMBDATA format is used for indexing and caching purpose and probably also contains thumbnails of processed images as well. They are generated so image previews (thumbnails) are quickly loaded on the display and users do not experience any sluggish behavior when they browse through their images, especially if they have a huge library.
The problem with these .thumbdata files is that they grow with each image the gallery/camera app encounters, and thus THUMBDATA indexes can even potentially store data about images that were deleted from the device. So a situation may happen, that even if you would delete all images from your storage, one or more .thumbdata files would still take up space.