Re: [PLUG] scripting help, variables in sed

On Fri, Nov 16, 2018 at 10:30 AM brent timothy saner <brent.saner@gmail.com> wrote:

On 11/16/18 10:02 AM, Michael Lazin wrote:
> Thank you, I am very familiar with find and exec, but this doesn't
> accomplish my goal. Maybe I wasn't clear. I want a number to be added
> in front of each file name and that number to be incremented by one so
> each file ends up with a unique name so they can be put together in a
> single folder. I was under the impression that sed -i would accomplish
> this, but maybe I'm doing this entirely wrong, I want to edit the
> filename and not the content. I am sorry, I am comfortable with bash
> but I am at best a noob programmer. Thanks.
>

ah, yes, i thought you were only interested in moving to a single
directory as that's the only thing explicitly mentioned.

so sure, if you want to do that:

__________________________________________
num=0
for f in $(find * -type f -name "*.jpg");
do
mv ${f} ${num}${f}
((num++))
done
__________________________________________

but that's a bit silly, that. they're already going to have a unique
filename if you're pulling them all from the same source directory.

the below not only guarantees a unique filename per unique file content
(MOSTLY[0]), but would have completely removed your need to run fdupes:

__________________________________________

for f in $(find * -type f -name "*.jpg");
do
newfname=$(md5sum ${f} | awk '{print $1".jpg"}')
mv ${f} newdir/${newfname}
done
__________________________________________

but again, this all predicates on you (pointlessly) running this on a
glob search against files which are all in the same directory.

if they're in subdirs, it's a little more comfortable to deal with in
python:

__________________________________________
#!/usr/bin/env python3

import os
import hashlib

#cnt = 0

for root, subdirs, files in os.walk('path/to/parent/dir'):
for f in files:
fpath = os.path.join(root, f)
if f.endswith('.jpg'):
# If you want hash-based naming, which has built-in dupe
avoidance:
h = hashlib.md5() # can also be a stronger hash, e.g. .sha512()
with open(fpath, 'rb') as data_in:
h.update(f.read()) # it'd be smart to implement
chunking here [1] but being that these are in theory just JPEG files...
new_fpath = os.path.join(root, '{0}.jpg'.format(h.hexdigest()))
# Or if you just want to use a blind counter prefix:
#new_fpath = os.path.join(root, (str(cnt) + f))
#cnt += 1
# Remember to uncomment cnt = 0 above.
os.rename(fpath, new_fpath)
__________________________________________

ta-da.

[0] https://en.wikipedia.org/wiki/MD5#Collision_vulnerabilities
using e.g. sha512sum for instance, while increasing the length of
the filenames, would avoid this. you're very much unlikely to encounter
it "in the wild" as a result of this particular use and scope, though.

[1]
https://git.square-r00t.net/OpTools/tree/centos/find_changed_confs.py#n68

___________________________________________________________________________
Philadelphia Linux Users Group -- http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion -- http://lists.phillylinux.org/mailman/listinfo/plug