- User Since
- Jan 1 2015, 12:53 PM (303 w, 2 d)
Jul 31 2015
Jul 28 2015
Ok, I feel with you. Haven't had such a fast RAID yet. Copying one evening worth of Arriraw (only OpenGate 75fps and 120fps 2.8K 16:9, no normal speed, all maximum) skater footage with 1,5 TB in a few hours was kind of hard to a 4 bay RAID5 TB-Array (was a fun project and no second target (expect of slow USB3 single drives) was available.
Jul 27 2015
Nope, my script reads two times. One for copies (coping in a buffer and writing to all destinations in parallel). And one time for checksum.
Jul 22 2015
Jul 21 2015
Well, Apple can be a bit of a pain sometimes but they are opening up ProRes more and more so maybe it is an option. A few years back there was no official solution for ProRes writing on Windows and now different applications are getting the go for example.
Jul 17 2015
I don't see rsync as a main need. Mostly it is a fast local transfer you need onset or so not the kind of network transfer or you use a network file system and can use the tool as if you are working local.
Jul 16 2015
Cool, I will keep an eye on your project an maybe find some time to help. Would be cool to finally have a nice data management tool. Have tried a few of the commercial ones and none of them really satisfied me so there is some space to fill.
Jul 15 2015
A few points that I see as important parts:
- 1:1 copy as mentioned earlier. (Maybe with an option to exclude .Spotlight, .Trash etc. when on a mac that sadly creates them directly when connecting the first time ...)
- Just have a list of targets where you can add from one to multiple (not only two) targets
- Parallel Copy: Read from source to RAM and copy to all target drives in parallel to speed up progress
- Parallel verification: Some simple tools do a checksum file(or to ram) from the source and check the targets then. Skip this and do checksums of source and all targets at once and compare then.
- add a project mode where you can set the target drives and directory scheme for this project that can be loaded as a starting point
- variable based directory scheme so that you can have something like "/media/irieger/MY_EXTERNAL_DRIVE/OperationApertus/Footage/%Y-%m%-%d/%camNo"
- an option to add another transfer to a queue that fill be processed when the current is finished
- an option to start a parallel file transfer (what I think about here is if you need to copy a small card from the sound guy who wants to finish in the evening and not have to let him wait until all the RAW footage is backed up or something similiar)
Would be nice if the user can select a checksum method and the checksums are then written to a file for manual checking (for example a md5sum-file that can be checked with md5sum -c $file. Checksum algorithms I'd really like to see are md5sum an xxHash (https://code.google.com/p/xxhash/), which seems to be much fast so could be helpful when transfering huge amounts of 4K raw sequences.