Featured image of post Bash shell script for removing duplicate files

Bash shell script for removing duplicate files

For removing duplicate files, I find this script to be useful.

#!/bin/bash
OUTF=rem-duplicates.sh;
echo "#!/bin/sh" > $OUTF;
find "$@" -type f -exec md5sum {} \; |
sort --key=1,32 | uniq -w 32 -d --all-repeated=separate |
sed -r 's/^[0-9a-f]*( )*//;s/([^a-zA-Z0-9./_-])/\\\1/g;s/(.+)/#rm \1/' >> $OUTF;
chmod a+x $OUTF; ls -l $OUTF
comments powered by Disqus
Built with Hugo
Theme Stack designed by Jimmy