Re: [PLUG] small business server virtualization?

[Rich Freeman <r-plug@thefreemanclan.net>]

On Tue, Nov 7, 2017 at 6:58 AM, Lee H. Marzke <lee@marzke.net> wrote:
>
> A good business case is NetFlix which runs on Amazon.   They  have a special
> service 'chaos monkey' that randomly kills ANY of their servers, doing this
> all the time, just to test that their recovery works.    Also Netflix runs
> the  "Directory listing"  on amazon, with the actual streaming of movie
> content from their private data center.     As you would guess, the
> performance and reliability for streaming just doesn't work on Amazon.
>

While I'll agree with just about everything you've said about Amazon
I'm skeptical of your last sentence, unless this is actually based on
something Netflix has said publicly.

I am guessing that performance/reliability isn't their main concern
for streaming so much as cost, and maybe some kind of licensing
conditions.

Reliability/performance shouldn't really be an issue, because
load-balancing that across many servers should be pretty easy.  If an
odd server goes down worst case a dozen users get a blip in their
video, which is probably fine if it is rare.  And if they cared that
much about it they could design with that in mind, and probably
already have to do so with their own data centers.  Performance
requirements for video streaming are pretty low unless you're
transcoding in realtime, which I doubt Netflix would have to do.
Really the processing power needed to stream the video is even less
than what your Roku needs to play it back.

I mentioned licensing speculatively, because I have no idea what
constraints Netflix is under with the content creators.  If Hollywood
is paranoid and tells them that they need to be encrypting all their
storage with hardware decryption modules or whatever then that might
be something they just can't do with AWS.  This might or might not be
a factor.

I think cost is the main factor and it comes into play in a few ways:

Bandwidth is of course a HUGE factor.  I can only imagine what kind of
bandwidth Netflix consumes, and even if they centrally serve the
content they could almost certainly negotiate at least as good a deal
on bandwidth as Amazon could, and even a tiny bit of profit in
Amazon's pricing would be a huge loss for Netflix if the pricing was
the same.  I wouldn't be surprised though if Netflix actually uses
more bandwidth for streatming than all of AWS.

Then you get colocation.  Because of those bandwidth costs Netflix
will try to forward position their content closer to their customers,
ideally inside of ISP networks so that they don't go out over the
network, which lowers their costs even more.  That means they are
running their own servers whether they want to or not, which means the
benefits of giving this over to AWS is diminished.

Then I wonder about the actual hardware costs.  Streaming is the sort
of thing that is extremely low-CPU but fairly high on IO/etc, so I
imagine that at large scale there could be savings for optimizing your
hardware around this.  Spending $1000 on hardware optimized for
streaming video could very well go further than the more generic AWS
options (despite the fact that AWS also has different options for
optimization around CPU/RAM/IO/etc).

And of course there is storage.  I'm not sure how much of a cost that
actually is these days with the costs of storage coming down (the
costs of storage must be dropping faster than the rate new content is
produced).  However, it could still be quite a bit.  Running storage
pods or Ceph or whatever probably could be more cost-effective than
paying for S3 at large scale.

It sounds to me that Netflix identified the one component of their
design that has the largest opportunity for savings and decided to
optimize it.  That is a good overall strategy for anybody using AWS.
Use them to get up and running quickly.  Optimize later.  But
definitely keep in mind the caveats Lee pointed out - they're designed
around the new model where hardware isn't intended to be perfect.  You
can't just migrate an application from a mainframe to AWS and expect
no issues without a redesign.

-- 
Rich
___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug