Saturday, November 24, 2012

Directory syncing

Directory syncing, as well as finding time, has proven  to be harder than I expected three weeks ago. Short version: I failed in my first two week goal. 
But the project is alive and i have learned quite a lot of things. Right now the program can compare two folders prompting which files should be copied, which ones are ok and which ones should be erased. This functionality isn't finished, because it doesn't check modification dates yet.
I'm using filepath.Walk to traverse the directory tree. Two independent coroutines are launched and the main function reads information from both of them. As path.FileWalk returns files ordered alphabetically it's possible for the main function to know which action should be take with each file.
But this implementation isn't good enough for various reasons. 
First of all, filepath.Walk behaviour is perfect for copying and creating directories and files, but not appropriated for deleting files and directories. We must delete first files and afterwards their parent directory. This problem can be fixed by keeping a list of files to be deleted and performing the deletion at the end. But honestly, I don't think this would be a good solution. So, a more customized version of path.FileWalk is needed.
Secondly, once the program has evolved and the remote encrypted copy is made, it will be inefficient, if not impossible, to check the remote tree as we do now. A local database with information of the last backup made must be kept. The local program should verify files using this database. Looking after remote files will be a task of the remote daemon.
I'm finishing this implementation in order to detect more pitfalls and learn file manipulation in Go. When it's finished I will start work with the database.

No comments:

Post a Comment