Re: [PLUG] scripting help, variables in sed

[Fred Stluka <fred@bristle.com>]

Brent,

Nice answer!  Considered, thoughtful, detailed, accurate.
I like it!

It's a shame we don't have a mechanism in the PLUG list for
StackOverflow-style ratings for good answers and helpful
contributors.  I'd give you a "thumbs up"!

--Fred
------------------------------------------------------------------------
Fred Stluka -- Bristle Software, Inc. -- http://bristle.com
#DontBeATrump -- Make America Honorable Again!
------------------------------------------------------------------------

On 11/16/18 10:30 AM, brent timothy saner wrote:
> On 11/16/18 10:02 AM, Michael Lazin wrote:
> > Thank you, I am very familiar with find and exec, but this doesn't
> > accomplish my goal.  Maybe I wasn't clear.  I want a number to be added
> > in front of each file name and that number to be incremented by one so
> > each file ends up with a unique name so they can be put together in a
> > single folder.  I was under the impression that sed -i would accomplish
> > this, but maybe I'm doing this entirely wrong, I want to edit the
> > filename and not the content.  I am sorry, I am comfortable with bash
> > but I am at best  a noob programmer.  Thanks.
> ah, yes, i thought you were only interested in moving to a single
> directory as that's the only thing explicitly mentioned.
>
> so sure, if you want to do that:
>
> __________________________________________
> num=0
> for f in $(find * -type f -name "*.jpg");
> do
> 	mv ${f} ${num}${f}
> 	((num++))
> done
> __________________________________________
>
>
> but that's a bit silly, that. they're already going to have a unique
> filename if you're pulling them all from the same source directory.
>
>
> the below not only guarantees a unique filename per unique file content
> (MOSTLY[0]), but would have completely removed your need to run fdupes:
>
> __________________________________________
>
> for f in $(find * -type f -name "*.jpg");
> do
> 	newfname=$(md5sum ${f} | awk '{print $1".jpg"}')
> 	mv ${f} newdir/${newfname}
> done
> __________________________________________
>
>
> but again, this all predicates on you (pointlessly) running this on a
> glob search against files which are all in the same directory.
>
> if they're in subdirs, it's a little more comfortable to deal with in
> python:
>
> __________________________________________
> #!/usr/bin/env python3
>
> import os
> import hashlib
>
> #cnt = 0
>
> for root, subdirs, files in os.walk('path/to/parent/dir'):
>      for f in files:
>          fpath = os.path.join(root, f)
>          if f.endswith('.jpg'):
>              # If you want hash-based naming, which has built-in dupe
> avoidance:
>              h = hashlib.md5()  # can also be a stronger hash, e.g. .sha512()
>              with open(fpath, 'rb') as data_in:
>                  h.update(f.read())  # it'd be smart to implement
> chunking here [1] but being that these are in theory just JPEG files...
>              new_fpath = os.path.join(root, '{0}.jpg'.format(h.hexdigest()))
>              # Or if you just want to use a blind counter prefix:
>              #new_fpath = os.path.join(root, (str(cnt) + f))
>              #cnt += 1
>              # Remember to uncomment cnt = 0 above.
>              os.rename(fpath, new_fpath)
> __________________________________________
>
> ta-da.
>
>
>
> [0] https://en.wikipedia.org/wiki/MD5#Collision_vulnerabilities
>      using e.g. sha512sum for instance, while increasing the length of
> the filenames, would avoid this. you're very much unlikely to encounter
> it "in the wild" as a result of this particular use and scope, though.
>
> [1]
> https://git.square-r00t.net/OpTools/tree/centos/find_changed_confs.py#n68
>
>
>
> ___________________________________________________________________________
> Philadelphia Linux Users Group         --        http://www.phillylinux.org
> Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
> General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug

___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug