How much cloud in the ownCloud?

Saying cloud we think Dropbox, Google Drive, iCloud or similar. They are more or less cloud storages with extra services operating of stored data like web photo albums, in-browser document viewers, sharing data options and so on. Cloud storage in short is about keeping your data somewhere out there relying on provider’s infrastructure. It is more than data centers or online storage however.

Cloud solution promises that all your clients (or devices if you will) are in the cloud, which means all they have instant access and consistent view of data. For example document uploaded from desktop to the cloud is available on your laptop and smartphone even in offline mode and the same document modification done on smartphone propagates to your other devices in the cloud.

All these ideas are not new. Developers see it every day in a SCM tools. Modern solutions like git or svn give you out of the box data copies and synchronization between copies to see latest greatest versions nearly effortless. Is the cloud just facelifted SCM given to the masses? The main difference is you barely use SCM on smartphone or smart-tv, while your multimedia content is welcome on such devices. The other difference is the way you consume the data: SCM gives no damn about it, it is just bunch of files and folders, while cloud solutions rapidly grow with additional content-specific support, like managing calendars or contact addressbooks, with sharing some details in mind. Lastly none of SCM tools was designed to efficiently handle large files, like multimedia typically stored in public clouds.

When it comes to large files serious cloud solutions must optimize the way synchronization is handled. For example you update description of your movie (let say ID3 tag in MP4 video file) you would be disappointed if thousands of megabytes are copied again to all your devices, especially if you are using 3G/4G paid transmission. And most solutions do it right. In simple cases they keep files chunked comparing checksums and swapping only small modified parts, more sophisticated solutions rely on P2P mechanisms like bit-torrent. As most providers requires to upload data to their cloud first bandwith is concerned when synchronizing new large files. And most of them deal with it using local peer to peer connections as long as synchronized devices are in the same network.

The number of cloud storage providers is growing and makes the choice really complex as they offer similar features. Take a look on this report for a start.

If entry level free of charge storage is 2-5GB why even bother about ownCloud? As the name says clearly, ownCloud is your own installed and maintained cloud. There are two main reasons to do it: unlimited storage and privacy. While the first one is vague if you compare TCO of server hosting or ownership, the latter makes no doubt. Whatever you upload to the cloud never disappears on provider premises. Most providers due to government regulations have to be able to give access to your data e.g. in case of criminal investigation; such formal backdoor means potential of data compromise. People who do not like to disclose their data yet badly need cloud superior features may keep data in cloud encrypted at the price of limiting service to typical cloud storage disk as the cloud does not understand content. They can use TrueCrypt to create virtual disks based on large file and then synchronize such a file with the cloud (error prone approach as synchronization of not dismounted disk leads to data corruption) or use dedicated software to encrypt-decrypt data on the fly (proxy between your device storage and cloud storage), like BoxCryptor.

As the ownCloud hosting requirements are minimal (roughly web server with PHP and database) and I already have mini server 24×7 for this blog and in-house mirrored data storage I decided to give it a try. With the respect to all contributors to this most advanced open source cloud, I felt a bit disappointed by the way it is marketed. As of version 4.5 here is what I found:

• Single server instance. Does not match cloud term well, as it should provide more location transparency. Not a case for private use though.
• No data transfer optimization. Data transfer is based on HTTP protocol and PHP uploads. Consequences: complete file reload (upload/download) every time, maximum files size limited by PHP upload capacity (and memory of your server). Forget about blue-ray or trucrypt/ISO disk uploads.
• Feature missing mobile clients. There is plenty features that current android client (v1.3.12) is missing:
  • Cannot automatically synchronize folders (file by file configuration only).
  • Missing automatic checks on outdated files or folders (check without upload).
  • Cannot choose root directory location (always /sdcard/owncloud).
  • Cannot map multiple folders as in desktop client.
  • Cannot sync only selected or current folder.
  • Cannot exclude chosen directories from syncing (why needed – see next point).
  • Battery drain and lags when large files directory mounted. As syncing reads all directories, no exceptions, client never stops automatically and drains battery nearly 20% per hour on my OneX. If syncing starts with large/deep directories in your cloud first you can wait tens of minutes to see it updated (or just drain battery down to zero as it happened to me). Worst thing is that even app is shot down and marked “no automatic sync” it starts syncing in background sucking you battery unattended (except for visible spinning-wheel icon in status bar)!
  • No distinction of network type (on wifi could automatically sync up, on paid connection only check if outdated, with except to the only feature “upload pictures via wifi only”).
• Architecture pluggable but secured. Anyone can plugin extension called application e.g. calendar, photo album, etc. If plugin fails of application is misconfigured you got blank screen in web browser; happy logfiles digging. That sounds more like PHP weakness though.
• Large files set indexing nightmare. Mounting new large local folder with 6000 files leads to blocking user interface for over 30 minutes (as indexing of all files into os_fscache table is always triggered by UI, no background/cron processing), and it will happen for each user having access to mounted resource. No warnings, no progress bar, just freezing browser blocked on each request. Bad bad bad.
• Gallery is embarrassingly slow. At list in my case (15k photos, 39GB total) mounted as local folder. Every time I enter gallery it just freezes for minutes. Small folders (dozen of pictures) work better but not seamlessly (should be fraction of second, like in WordPress on the same hardware). Again, no warning, no progress bar, nothing. Still just a bunch of pictures its is not the goal of “gallery”, is it?
• Outdated user documentation. Project is evolving, fine, however some basic features (like mounting samba share as external storage) are cryptic. Requires googling and PHP code digging to figure it out.
• Half-cooked features. ownCloud has many features, unfortunately many of them are incomplete from user perspective. Walking thought the features to see what is working and what is not I discovered plenty nice (sometimes basic) suggestions. Problem is they are waiting long months to be implemented.

Even though ownCloud sounds more like steam burst from chimney than cloud in the sky, with management of large files/folders sets being a nightmare, I am still two-thumbs supporter of this approach as I want to keep my data on a leash.