0% Complete
0/42 Steps

Shared Storage & Collaboration

Lesson Progress
0% Complete

Shared Storage (Active Storage)

Shared Storage with Phil Crawley

Avid Unity was the first shared storage. SNFS is the common file system.

Start here to get right to Terablock, the first “cheap shared storage system”. It was a block-level SAN mounted as a fixed read/write disk on a single workstation. Nowadays it basically mimics an Avid media composer storage (unity or ISIS) system so you can use AVID’s shared storage features. Historically it mounted on ohters as a read-only disk but I think that’s changed.

Moved from fibre channel SANs to iSCSI via Ethernet-based SANs. This is an IP-based system 

We now use compression DNXhd 185 is 23mB/seconds and uncompressed SD was 21MB/s. Compression has allowed us to get better bang for buck video quality in smaller payload. Cat 5/6 ethernet can carry it.

Clients run an iSCSI “initiator” which makes the storage look like a mounted disk.

TCP/IP based protocols like SMB (samba), AFP, aren’t optimized for video. A NAS is clever in how you can upgrade it’s size, but it’s connecting using these industry standard network protocols.

EditShare

LumaForge Jellyfish

SNS EVO

Nowadays it’s most cost-effective (under $5k) to just configure a 10Gbe NAS for small teams of shared storage users. The bulk of the cost is then in the server-quality HDDs. Bob Zelin (bobzelin@icloud.com) has a good post on Creative Cow and it’s worth your time to simply contact him if you’re ever in the market for this.

SSDs have shorter lifespans in server situations.

Multiple people will need access to the data at once in a post facility.


All in all it is a solid recommendation from my side. “QNAP’s work wonderfully as shared storage systems for RED workflow. To be clear, they work perfectly, and very fast for Adobe Premiere, FCP X, Davinci Resolve, and even AVID Media Composer (if you install Indiestor Mimiq on your editing computer). With a 1G connection, you will get 100 MB/sec bandwidth. With a 10G connection (using a thunderbolt to 10G adaptor from Promise or Sonnet), you will get 800 MB/sec – more than enough for 6K and 8K workflow. While the TVS-871T is a wonderful inexpensive product, and it’s “replacement” is the newer TVS-1282T, the best current value from QNAP is the incredible TS-1685, which is a 12 bay, that has built in 10GbE ports, and can be expanded with 4 additional expander chassis. To be clear, if you install 10TB HGST NAS drives in a TS-1685, after a RAID 6, you will have over 100 TB of usable storage in this tiny box, and all the editors will be able edit 4K, 6K and 8K. For multiple systems, as suggested above, get a small 10G switch from Netgear – specifically the XS708T or XS716T. Great products, easy to setup, and lifetime warrantee. 

Bob Zelin”

The QNAP TVS-871T, TVS-1282T, and TS-1685 all come with the dual 10Gbase-T ports built into them, for free. You don’t need to second source anything – it’s all FREE from QNAP. The TVS-871T will work wonderfully for you, for your editors who are working in 4K. If you only have 2 computers, you don’t even need a switch – you can just go into the two built in 10G RJ45 ports and edit away. You must assign static IP addresses for these on two different subnets if you don’t use a switch. Do not get the TS-453B. You need EIGHT DRIVES to reliably do shared storage for multiple editors – particularly if you are doing 4K media, and ESPECIALLY if you are going to do a multiclip or multicam job. You are correct that thunderbolt 3 can only be a very short cable, and will reduce to thunderbolt 2 speeds if you extend the cable. And you will find that thunderbolt 2 for video editing is pretty unreliable (this connection is called thunderbolt bridging). You are much better off getting a thunderbolt to 10G adaptor for your computer (the Promise SanLink3 for thunderbolt 3 is only $299 US retail), and then you can plug directly into the 10G port on the TVS-871T, and you will have super fast speeds for 4K and 6K editing.  The price of the entire QNAP system for doing shared storage is a fraction of the price of any of your RED gear !!!!! Remember, you can only run a thunderbolt 3 cable 2 meters. Any longer and it drops in speed. With the 10G adaptor, you can have a 55 meter Cat6 cable on it that costs you 10 bucks ! In the past I only used AFP, but now I am only connecting with SMB. I turn off SMB Signing on every Mac with the nsmb.conf file. It’s a pain, but it makes a huge difference in performance. I used to use NFS before FCP X 10.3, because you could not write a library to the QNAP unless you used NFS, but now that Apple fixed this, I only use SMB.  For new systems, with Adobe Premiere, or Resolve, I use SMB. I face the reality that Apple wants AFP to go away. You do not need to buy a special cable (cross over cable) to go into the QNAP directly from your Mac or Win PC system. You plug directly into the QNAP ports (either the 1G or 10G ports) and assign static IP addresses on different subnets and you will have no problems. You can have 2 10G clients, and 4 1G clients with no switch, if you like. The switch just makes it easier, and you can keep everyone on the same subnet. 

https://www.reduser.net/forum/showthread.php?162272-Qnap-raid-system/page11

Archive

NAS

This refers to “Network-Attached Storage” as opposed to “Direct-Attached Storage”. The former is a nice archive solution, but generally the latter is required for an actively working drive. However, some of the direct attach options (e.g. TB3) available on a NAS make it viable for field use as well.

Synology

QNAP (e.g. TVS-882ST3 or Bob’s recommendation TS-1685)

Synology

I use a Synology RS3614xs+ a 12-bay, 35TB usable, and at home I use a DS1515+

Synology Hybrid RAID

Quad Core Xeon processor with enterprise features like dual power supplies and swap-ability. You can also buy a 10Gb Ethernet adapter for it looks like.

Indexed Database

  • Excel Spreadsheet
  • YoYotta
  • CatDV

Drobo vs UnRAID?

Drobo comes as DAS or NAS, but caps out at 12 bays or 128TB max.

UnRAID is like a more customizable, easier-scalable NAS, but similar to Synology and QNAP. With unRAID, you can start with a single drive and grow it with an array of random, mismatched drives from there. With Synology, you can swap an old drive for a new, larger, one, allow it to rebuild, and one-by-one upgrade them all. Or you can get a $500 5-bay external expansion bay.

“The people who get the most out of it are probably the ones doing a lot more with it – Dockers, applications, VMs, etc.”

Tape (LTO) vs Hard Drive

  • Cost
  • Longevity
  • Linear vs Random Access
  • Upfront Cost

There’s a lot of opinions on it, but basically powered off disks are generally considered less reliable than tape. If they aren’t being used, HDDs are more subject to failure. They’re physically larger, have moving parts, and are more pricey than LTO. LTO-8 provides 12TB uncompressed and 30TB of compressed data capacity on a single cartridge priced around $100. But, bear in mind that LTO’s costs are quite front-loaded. You’ll need to buy the drive ($2k+)and the tapes and software. LTO tapes are rated at 30-yr longevity. LTO advances quickly and is only backwards compatible up to two versions. It’s linear, tape-based storage so there’s no skipping to the middle of it quickly to overwrite an old file–essentially you keep the old file and just write a new one to the end of the tape.

“With the introduction of LTFS, The LTO appears as a device and allows you to drag and drop files to and from the tape. In short, it allows the LTO drive interface to function like a regular hard drive. This removed the need for any special software and made saving and copying files much more intuitive for the user. It also has the added advantage of making LTO drives compatible across all the major operating systems.”

The Cloud

Scroll to Top