Re: [PLUG] scripting help, variables in sed

[John Von Essen <john@quonix.net>]

Im waiting for the Perl example….

-John

> On Nov 16, 2018, at 10:58 AM, Fred Stluka <fred@bristle.com> wrote:
> 
> Brent,
> 
> Nice answer!  Considered, thoughtful, detailed, accurate.
> I like it!
> 
> It's a shame we don't have a mechanism in the PLUG list for
> StackOverflow-style ratings for good answers and helpful
> contributors.  I'd give you a "thumbs up"!
> 
> --Fred
> ------------------------------------------------------------------------
> Fred Stluka -- Bristle Software, Inc. -- http://bristle.com
> #DontBeATrump -- Make America Honorable Again!
> ------------------------------------------------------------------------
> 
> On 11/16/18 10:30 AM, brent timothy saner wrote:
>> On 11/16/18 10:02 AM, Michael Lazin wrote:
>>> Thank you, I am very familiar with find and exec, but this doesn't
>>> accomplish my goal.  Maybe I wasn't clear.  I want a number to be added
>>> in front of each file name and that number to be incremented by one so
>>> each file ends up with a unique name so they can be put together in a
>>> single folder.  I was under the impression that sed -i would accomplish
>>> this, but maybe I'm doing this entirely wrong, I want to edit the
>>> filename and not the content.  I am sorry, I am comfortable with bash
>>> but I am at best  a noob programmer.  Thanks.
>>> 
>> ah, yes, i thought you were only interested in moving to a single
>> directory as that's the only thing explicitly mentioned.
>> 
>> so sure, if you want to do that:
>> 
>> __________________________________________
>> num=0
>> for f in $(find * -type f -name "*.jpg");
>> do
>> 	mv ${f} ${num}${f}
>> 	((num++))
>> done
>> __________________________________________
>> 
>> 
>> but that's a bit silly, that. they're already going to have a unique
>> filename if you're pulling them all from the same source directory.
>> 
>> 
>> the below not only guarantees a unique filename per unique file content
>> (MOSTLY[0]), but would have completely removed your need to run fdupes:
>> 
>> __________________________________________
>> 
>> for f in $(find * -type f -name "*.jpg");
>> do
>> 	newfname=$(md5sum ${f} | awk '{print $1".jpg"}')
>> 	mv ${f} newdir/${newfname}
>> done
>> __________________________________________
>> 
>> 
>> but again, this all predicates on you (pointlessly) running this on a
>> glob search against files which are all in the same directory.
>> 
>> if they're in subdirs, it's a little more comfortable to deal with in
>> python:
>> 
>> __________________________________________
>> #!/usr/bin/env python3
>> 
>> import os
>> import hashlib
>> 
>> #cnt = 0
>> 
>> for root, subdirs, files in os.walk('path/to/parent/dir'):
>>     for f in files:
>>         fpath = os.path.join(root, f)
>>         if f.endswith('.jpg'):
>>             # If you want hash-based naming, which has built-in dupe
>> avoidance:
>>             h = hashlib.md5()  # can also be a stronger hash, e.g. .sha512()
>>             with open(fpath, 'rb') as data_in:
>>                 h.update(f.read())  # it'd be smart to implement
>> chunking here [1] but being that these are in theory just JPEG files...
>>             new_fpath = os.path.join(root, '{0}.jpg'.format(h.hexdigest()))
>>             # Or if you just want to use a blind counter prefix:
>>             #new_fpath = os.path.join(root, (str(cnt) + f))
>>             #cnt += 1
>>             # Remember to uncomment cnt = 0 above.
>>             os.rename(fpath, new_fpath)
>> __________________________________________
>> 
>> ta-da.
>> 
>> 
>> 
>> [0] https://en.wikipedia.org/wiki/MD5#Collision_vulnerabilities
>>     using e.g. sha512sum for instance, while increasing the length of
>> the filenames, would avoid this. you're very much unlikely to encounter
>> it "in the wild" as a result of this particular use and scope, though.
>> 
>> [1]
>> https://git.square-r00t.net/OpTools/tree/centos/find_changed_confs.py#n68
>> 
>> 
>> 
>> ___________________________________________________________________________
>> Philadelphia Linux Users Group         --        http://www.phillylinux.org
>> Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
>> General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug
> 
> ___________________________________________________________________________
> Philadelphia Linux Users Group         --        http://www.phillylinux.org
> Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
> General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug

___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug