NAS and DLNA

[rplace]

So I got a million files all over the place. I have ripped my large-ish CD collection to FLAC, WAV and two bit rates of MP3. Then I lost it all due to drive failure (I know back it up). Finally I have them re-ripped. My current quest is for a decent NAS running RAID 1 or RAID5. Looking at some QNAP and Synology device for the NAS. Right now my quest is for managing music, but I know that will eventually lead to video so a DLNA server is on my radar.

Has anyone taking this task on? Can you give me the Cliff Notes version? What do you like what don't you like. Is a TB of storage enough currently or should I be looking at 1.5 to 2 TBs? How important is CPU speed and Memory size on the NAS for quality streaming? Budget boxes look to typicall have a 800mhz processor and 128/256 of ram...while all the serious devices seem to be 1GHZ or higher and all have at least 512 of RAM. I want to do this once and do it right, with no regrets, but I don't want to waste money on features I won't use. What are your thoughs on buying a NAS with hard drives versus buying as discless system and getting the drives myself. I'd guess the large resellers buy in volume so a system with discs might be more value.

[Olorin]

I haven't done this yet, but one thing I can say for certain is . . . NERDS!!

[dtel]

I haven't done this yet, but one thing I can say for certain is . . . NERDS!!

Yep it's called the Klipsch forum. [8-|]

[rplace]

I haven't done this yet, but one thing I can say for certain is . . . NERDS!!

Thanks for the link to the post O-man! I posted in his thread will see if I get any answers. I really want somebody to read my mind and just tell me what to buy so I can stop obsessing.

[michael hurd]

I haven't any experience with NAS or DNLA, but I can tell you that if you want to dabble in video at all, you will want at least 4 TB. Considering each ISO backup ( DVD ) I have made is between 4-7 GB on average, and two discs could take upward of 14GB, it doesn't take long to gobble up room.

Transcoding to smaller file sizes ( I have been making some MKV's with DD AC3 sound, DTS when available on the disc, and also MP3 audio, with H.264 video ) takes an awful lot of computing horsepower. ( Think Intel I5 series or better processor ) My laptop with a dual core AMD Turion X2 can take between 4-5 hours to transcode one 4 GB ISO to MKV. I5 should cut that down to around a half hour.

Considering that you can get a 2TB bare drive for around $ 115 or so, it doesn't make fiscal sense to go with anything at all smaller.

Bottom line is that you need to plan for what you want to do with your collection. If you want to stream movies to more than one connected device at once ( think two client boxes like the Popcorn Hour etc... or WDTV LIVE ) you may want to ask around for processor requirements etc.

I know a fellow who built up a 40TB server for his HOME, and he's got it about half full at the moment. [:|] BluRay rips take up a lot more space than a regular DVD.

If you are going to go more than 4TB, I would consider building a rack mount server with dual processors and a lot of storage bays.

[Olorin]

You're welcome Rich! I hear ya on the getting it done part too. I've looked at these kind of casually and I gravitate toward a two-bay D-Link or Buffalo Technologies enclosure that will do NAS, DLNA, iTunes server, and torrents. I'd love a larger unit that can do RAID5, but the price from two-bay to three seems to take a disproportionate leap, and then going larger starts getting into real business hardware costs, so I wonder about just mirroring two 1.5 TB drives and calling it a day. That's still a few hundred bucks all-in though, and I know it will prove itself inadequate and not-scalable in the space of a year or less, so all this dithering means this project just hasn't climbed up the priority list high enough for me to stop wondering and start buying. I'm sure I'll readjust my thinking as soon as the 300 GB drive in my Mac Pro that holds all of my 1200 or so albums in Apple Lossless decides to crap out.

[rplace]

Considering that you can get a 2TB bare drive for around $ 115 or so, it doesn't make fiscal sense to go with anything at all smaller.

Michael thanks for the info on DVD rips and size. Currently I am DVD only no BRay or HD. I hear you on the space issue. The reality is that discs cost a lot more for a RAID NAS system. You can, or should not, use desktop type hard drives. I see 1TB drives in the 85 to 250 dollar range...and it is the higher end drives you need for RAID. So figure 150 or so for a single TB not 2. No figure you need 3 and better yet 4 drives. The cost really adds up quickly.

I think my primary need will be hight end audio and digital pictures. I'll probably play around with streaming video, but to be honest it really is not all that much hassle to grab a DVD and put it in the player when figure you are investing 2 hours or so. I also want it for back up of my personal and business computers. I have been more or less lucky to date, but that can't last forever. I have a single 500GB Mirra personal back up system from many years ago so I think it is time to be realistic and think about its replacement. Storage has gotten crazy cheap from years gone by, but when you are talking multiple HDDs it still adds up. Still it would be very handy to not have any media storage and throw all those cases in offsite storage for the ultimate backup. Wait, I have to keep my vinyl handy, but that is fun the drag out and hold.

[rplace]

...I gravitate toward a two-bay D-Link or Buffalo Technologies enclosure that will do NAS, DLNA, iTunes server, and torrents.

Based on another post I read here it was the Buffalo devices that got me hot on this. After just briefly looking on the Interwebs I quickly changed my tune. Seem nobody like them. The attractive thing was the price. Seem like this might be one of those cases where you get what you pay for. At a minimum get yourself a USB drive and back your music up. For less than $100 that is cheap insurance. I was so bummed when I lost all my music. What a pain to redo it all.

[michael hurd]

Considering that you can get a 2TB bare drive for around $ 115 or so, it doesn't make fiscal sense to go with anything at all smaller.

You can, or should not, use desktop type hard drives. I see 1TB drives in the 85 to 250 dollar range...and it is the higher end drives you need for RAID. So figure 150 or so for a single TB not 2. No figure you need 3 and better yet 4 drives. The cost really adds up quickly.

Why is it you shouldn't use desktop drives? I have read that WD disabled TLER in the newer Black drives. I guess they want you to buy a RE drive instead. I haven't checked into the other brands yet, but someone must have inexpensive drives that you can use in a RAID array.

[rplace]

Why is it you shouldn't use desktop drives? I have read that WD disabled TLER in the newer Black drives. I guess they want you to buy a RE drive instead. I haven't checked into the other brands yet, but someone must have inexpensive drives that you can use in a RAID array.

Michael, I am a software developer by profession and admit don't know a whole lot about hardware...and actually I like it that way. I find all the nuts and bolts of machines and networks quite boring. That is why I am trying to learn something here rather than reinvent the wheel. I can't give you a good answer but I seem to remember it has somethign to do with Time Limited Error Recovery and the RAID controller. In short Desktop drives are not all the critical and Enterprise drives are just built to a different standard. I wish I could give you a better answer, but I am now of the mindset (read brainwashed [] ) that I need something beyond the cheapest drive to really be secure. I can't tell you how bummed I was when I lost all the music. It was not that it was gone because I still have all the CDs. It was the time invested to get it there. I seriously considered one of those services where you send off all your CDs and they rip them for you.

Also I think the WD Black drives are some of the "better" drives and can be used in RAID application IIRC.

[psg]

The reality is that discs cost a lot more for a RAID NAS system. You can, or should not, use desktop type hard drives. I see 1TB drives in the 85 to 250 dollar range...and it is the higher end drives you need for RAID.

I have never heard that one before. Why would that be? What makes RAID so special? If anything, a disk in RAID5 may work LESS than a standalone disk since it isn't getting all the data.

[Olorin]

...I gravitate toward a two-bay D-Link or Buffalo Technologies enclosure that will do NAS, DLNA, iTunes server, and torrents.

Based on another post I read here it was the Buffalo devices that got me hot on this. After just briefly looking on the Interwebs I quickly changed my tune. Seem nobody like them. The attractive thing was the price. Seem like this might be one of those cases where you get what you pay for. At a minimum get yourself a USB drive and back your music up. For less than $100 that is cheap insurance. I was so bummed when I lost all my music. What a pain to redo it all.

That's good information! I was looking at the Buffalo just because I use their broadband router with the Tomato firmware and it really, really rocks. Download rates are 4x the old Linksys on the same connection . . . shame their proficiency in the broadband router apparently didn't carry over into the NAS box.

For a while I've used a FireWire drive to carry stuff from home to work, and work is indulgent, so I have a 1 TB external drive on the Mac here that has everything on it that the home Mac has . . . everything that doesn't violate work's AUP, anyway . . . so I do have some form of backup.

I think I may just software mirror a couple of 2 TB drives inside the Mac Pro case. It's orders of magnitude better than what I have, and the drives can always go into a whatever-box later on down the road.

[rplace]

I have never heard that one before. Why would that be? What makes RAID so special? If anything, a disk in RAID5 may work LESS than a standalone disk since it isn't getting all the data.

I don't know, like I said earlier this is not my area of expertice, that is why I am looking for some advice from people that know more than me. I have read it in more than one place. Do a little searching and I am sure you will find plenty of info. Below is something I had found from a company that makes RAID devices and does NOT sell hard drives. It make sense to me that if you are a company like DELL and you are spitting out many many desktops then saving money and cutting corners is paramount. If you are running mission critical data center I would expect better hardware than I have sitting on my desk. Kind of like all of us here having mega-$$$ speakers/amps/source while our neighbors are happy with the speakers in their TVs. We demand just a bit more[] At this point with all the time I have invested in the data, and what I want from the box (performance and reliability) I'm not going to worry about an additional 50 bucks per drive. To keep the cost down initially I am now leaning toward a 4-bay NAS but only getting two drives and doing RAID 1 to start. Then I will add a 3rd and 4th drive as I need it.

According to hard drive manufacturers, desktop class hard drives are designed for balanced performance and reliability for everyday computing, while enterprise class hard drives are designed to deliver higher performance standards suitable for 24x7 mission critical business applications. Hard drive manufacturers do not recommend installing desktop class hard drives for the RAID environment regarding to these reasons. Please choose the hard drive that fits your needs.

[psg]

Well, I use off-the-shelf OEM drives at work on my Linux server, which stays up and running 24/7. Drives don't fail at abnormal rates; maybe one of the four every few years. The beauty of redundant RAID is that one drive CAN fail without losing any data, even using cheaper disks.

Industry run what are called mission-critical applications where just the inconvenience of any down time to rebuild a RAID array after a disk failure means a financial hit. Paying a few hundred more in that case is peanuts. Home use has quite different implications.

[Ironsave]

Well, I use off-the-shelf OEM drives at work on my Linux server, which stays up and running 24/7. Drives don't fail at abnormal rates; maybe one of the four every few years. The beauty of redundant RAID is that one drive CAN fail without losing any data, even using cheaper disks.

Industry run what are called mission-critical applications where just the inconvenience of any down time to rebuild a RAID array after a disk failure means a financial hit. Paying a few hundred more in that case is peanuts. Home use has quite different implications.

I currently maintain 4 sorting machines and perform backups to 18 computers on a quarterly basis for these machines.

I have had one fail so far, but we replace them with the same type of drive that was there.... and all are available at New Egg or Tiger Direct....

Heat and dust are the two big killers of any PC or hard drive (really any electrical component). The enviornment in this facility exceeds 115 degrees in the summer and is extremely dusty.

I have a Raid Array running at home just for speed on my PC. (AMD Phenom II). All music and movies are on an external 1 TB drive. (Western Digital "Black Box"). I am currently with about 350 GB free. My music (Mostly 320kps) fills about 190GB.

I am about to migrate music to another drive and keep movies and music seperated.

The blue ray movies can take anywhere from 5-18GB depending on if it is a direct burn or a rip.

I agree, plan for the future, 4 TB minimum.....

[eq_shadimar]

Well I may just be able to help out here a little bit since I have been doing this for several years now, and I am a hardware guy []

First in a home application desktop drives in a RAID configuration are fine. The actual RAID class HDD's are meant for business applications (think Google's or eBay's data center) and are way overkill for streaming music and video files around your house.

Second if you are going to want more than four drive bays then you will want to build your own server. I recommend unRaid on Linux. Here is a link to their site: http://www.lime-technology.com/

Third whichever way you go buy the most amount of storage that you can afford. Many NAS boxes and unRAID will let you add capacity and grow the RAID on the fly without having to do a complete backup and rebuild but it is a long process. If your RAID solution cannot do this then whenever you want to add space you will need to totally backup all your data, rebuild the entire array with the additional harddrives, and restore the data (ick).

Ok so what am I using? I have a Promise SmartStor 4600n with 4x1 TB Drives for a total usable storage space of ~3TB. With In a RAID5 configuration you will always lose the space of one drive for RAID overhead. This overhead is what allows the RAID to be rebuilt if you lose a single harddrive. You need to have at least 3 harddrives for RAID5. So in a 4 bay enclosure with 4 1TB drives you could have the following configurations:

RAID5 with hot spare ~2TB (a hot spare is a harddrive that sits in the enclosure doing nothing until a harddrive fails. The RAID will detect the failure and automatically start rebuilding the RAID to the hot spare without you having to do anything.

RAID5 ~3TB

JBOD - Just a bunch of disks - ~4TB, no redundancy, no spare, if a drive fails you lose all the data that drive contained.There are some other configurations such as a mirrored RAID1 which would give you ~2TB of space as well.

As far as the processor required I think that any NAS you purchase will be fine streaming a single stream. You only have to worry about CPU horsepower if you want to stream HD content to multiple clients or endpoints.

Ok now on to DLNA. DLNA is a great idea. You can put a tiny client inside a TV or whatever and have it stream the media from the DLNA server. In practice it does not work so well. The biggest issue is that while your DLNA server may be able to see and stream DVD ISO’s, FLAC files, etc most DLNA clients imbedded in TV’s and such will only be able to use MP3, MPEG4, and JPG files. A DLNA client in a media streamer like a Popcorn Hour or XBMC will be able to accept most media files. Bottom line you want to look for a streamer that will support SMB (Microsoft drive mapping standard) for the most flexibility. Using SMB your media on the NAS will appear as a network drive or resource on your media streamer.

I use Popcorn Hour A-100’s and I am planning on getting either the Popcorn Hour C-200 or a Dune streamer for Blu-Ray ISO’s. There is no perfect media streamer at this point as far as I am concerned. They all have their faults. Most streamers are great with video but the music interfaces are lacking. I have loaded MPD and MPDJB on my A-100’s and the music interface now is much better but you will have figure out how to get them installed and working. Not a problem for me.

If you have any more specific questions e-mail or post here and I can try to answer them.

Laters,

Jeff

[rplace]

All great info, Jeff, thanks! Right now I am leaning toward a QNAP TS-419P. I think I will start with 2X2TB drives in RAID 1 and add the 3rd and 4th drive a funds permit. It does allow drives to be added on the fly. That is a nice feature. Got any thoughts on that device? Also on the short list is a Sysnology 410, but I like the ease of removing drives on the QNAP, though I don't expect to do it much. Have read good things about both.

Thoughts?

[eq_shadimar]

QNAP is one of the best out there but really either one should work great for you. If you don't already know about it Small Net Builder is a great site for network equipment (NAS, Routers, Switches, etc) reviews: http://www.smallnetbuilder.com/

One other important item that you will need is a good way to manage your music. I have around 67,000 music files and the only application that will manage that many files that I like is Media Monkey: http://www.mediamonkey.com/

It has great features for tagging and fixing the tags on all your music files along with all the normal stuff you would expect (playlists, sync with portable player, etc).

Laters,

Jeff

[psg]

Lots of great info Jeff! Thanks!

[Guest " "]

raid1, 5, 10, etc, won't protect your files from directory and partition problems. You need both a hardware based drive management system and an a software based directory or partion management system.

for the directory and partiontion management you can use various disk cloning or backup solutions.

do not make the mistake that hardware redundantcy will protect you from directory or partition issues.