Re: [PLUG] AWS Scripting using awscli and bash

On Fri, Mar 16, 2018 at 8:54 AM, Andy Wojnarek <andy.wojnarek@theatsgroup.com> wrote:

Great read, thanks for sharing.

In my presentation, I have “unlimited scale” in the pros column, but maybe I should have included it in the con column too. I saw this on reddit:

s3 fiasco - 100+ million duplicated files in a versioned bucket
“So, we built a lambda that handles image compression when images are dumped in a bucket, but didn't handle tagging properly and it ran in a loop for a few hours and stacked a painful amount of duplicate objects. The quantity is so massive at this point, everything I've tried with the command line tools/boto has just crashed - and the bucket is versioned so I'd have to loop through and delete every version of every file. Anyone have any great ideas? The data prior to the run date needs to be retained.”

Uh oh…. Pretty easy to generate enough usage to bankrupt a project if you aren’t careful…. With an on-prem solution, you can only scale within your limits of what you’ve purchased.

--
Andy

On 3/14/18, 7:39 PM, "plug on behalf of JP Vossen" <plug-bounces@lists.phillylinux.org on behalf of jp@jpsdomain.org> wrote:

$20
https://www.amazon.com/AWS-Scripted-Automate-Deployment-Resilient/dp/1503137775/
but see http://www.quickstepapps.com/ and download the code from
http://www.quickstepapps.com/wp-content/uploads/2014/10/awsscripted.zip.

I first read this book at the end of 2016 but I had no context for most
of it. I just re-read it today, in light of Andy's certification preso
and some stuff at $WORK, and I thought it might be of interest.

This is a book that automates 100% of deploying "a resilient Web
Application" in AWS, using the `aws` cli and bash from a Mac laptop.
The author uses a Mac because it has bash and Windows sucks, but it's
not clear why the thought of using an actual Linux desktop never seems
to have occurred to him. (Shut up Walt. :-)

It's in...bash. That's cool because...hey, bash! But in a way it bugs
me because Ansible is really a lot more suitable to a lot of what he's
doing. There's also a lot in the author's bash code that I don't like,
and there are some questionable security practices and pretty awful uses
of `expect`.

BUT...it's bash. It's easy to see what is going on, and the examples of
doing really practical things are REALLY, REALLY nice. It seems there
are a lot of things you have to do in a certain order, and you need to
build on previous steps using IDs of things you just created. And
that's all in here, in code.

I haven't tried any of the code yet because my API access isn't sorted
out, but I have no reason to believe the code will not work.

There's enough "book" there to get you going, but most of the later
chapters are a couple of paragraphs and then just the verbatim code...
The AWS code is interesting, but it gets into a lot of PHP, Apache and
other stuff I don't care about. Still, the practical example is a fine
idea and I like it.

Enjoy,
JP
-- -------------------------------------------------------------------
JP Vossen, CISSP | http://www.jpsdomain.org/ | http://bashcookbook.com/
___________________________________________________________________________
Philadelphia Linux Users Group -- http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion -- http://lists.phillylinux.org/mailman/listinfo/plug

___________________________________________________________________________
Philadelphia Linux Users Group -- http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion -- http://lists.phillylinux.org/mailman/listinfo/plug