Hi,
Yes it is currently neccesary that the Data folder be writable unless you don't need to upload images and other files and don't need the search engine. Files are stored on disk under webroot/Data/Sites/[SiteID]/ so in a single site installation site specific files are under webroot/Data/Sites/1/
If your organisation would be interested in sponsoring development of features to allow everything to come from the database please let me know. I would be gald to implement that but its not a high priority unless I get a customer who needs it and is willing to pay for development.
Note that the data folder does not contain the database, it only contains files meant to be writable. For example in addition to gallery images, skins are located there. Skins consist of a master page and css files and images, ultimately I plan to have a browser based editor for creating skins. So in this example even if we had database file storage the master page would typically be installed on the file system even in a farm so either we would not use that feature or we would have to write code to load master pages from the database instead of the file system. The purpose of the Data folder is to have a known writable footprint, no other folders or files need to be writable, anything writable must be beneath the data folder. Depending on the size of the farm there are ways of sharing or keeping in sync a tree of files. The perfomance in just letting IIS serve images and static files from disk I think will always be faster than running managed code to get every image on the page but in large farms I can see where the everything should be in the database idea becomes more attractive. I'd love to build that but its not something I or my customers have needed yet and its a lot of work and has its own set of tradeoffs vs using the file system.
Best Regards,
Joe Audette