Is there a good program I can use to...

Drez

...compare and see if I have duplicate files on my PC, or in two specific locations/drives on my PC? Something shareware/freeware preferably.

Nocren

Do a search and compare file locations?

Lord knows I something similar for all my damn MP3s and pictures.

Mugenmidget

I see there's a few on Google, but I can't vouch for any of them because I don't have much experience (but I want something similar as well).

This is the one I'm going to try right now: http://www.tucows.com/preview/373411

But it'd be nice if someone with prior knowledge could suggest something awesome.

randombattle

Jesus I was just about to look for one of these myself....

Creepy.

I'm giving that program a shot too.

Fellhand

Drez wrote: »

...compare and see if I have duplicate files on my PC, or in two specific locations/drives on my PC? Something shareware/freeware preferably.

You mean like mirrored drives or like drives that synch?

Doc

Fellhand wrote: »

Drez wrote: »

...compare and see if I have duplicate files on my PC, or in two specific locations/drives on my PC? Something shareware/freeware preferably.

You mean like mirrored drives or like drives that synch?

or check for duplicate porno

peterdevore

If you're just worried about 'media' files, you could search for *.jpg, *.wmv, *.avi, *.mpg and so forth and then sort the gigantic list you get on file size or name by clicking on their column header. Files named the same or with the same size will show up next together and then you can easily see which are duplicates.

stranger678

I have managed to accumulate more then 6000 duplicate mp3's so if you guys find any automated dupe finder in your quest here, please let me know. Because deleting them by hand for the last week is turning into one bitch of a chore.

Einhander

stranger678 wrote: »

I have managed to accumulate more then 6000 duplicate mp3's so if you guys find any automated dupe finder in your quest here, please let me know. Because deleting them by hand for the last week is turning into one bitch of a chore.

I don't mean to derail this thread but how in the fuck?

stranger678

Between Me, My wife, and My roomate, we have something like 60g of music, when we change computers, we do backups and saves, and then combine crap together on the new PC's, my newest rig, my music folder has swelled to like 80g, because of dupes from all the copy and paste backups.

CZroe

I often disable the OS zip support and search the HDD for *.* and "large files over 1MB" using Windows' search. I view by details and sort the results by filesize and the largest dupes stick right out.

templewulf

Finding duplicate files by OS search only compares filenames. Does the thing on tucows compare CRCs or anything? 'cause I know we have duplicate porno (with different titles), but I don't want to watch it all to find the dupes.

I mean, I do, but I tend to have to wash my hands a lot, and I forget which ones are which when I come back.

CZroe

Like I said, organize the results by filesize and the filename doesn't matter. Just confirm that they are the same using common sense.

Veevee

CZroe wrote: »

Like I said, organize the results by filesize and the filename doesn't matter. Just confirm that they are the same using the dirty hands method.

;-)

Doc

Finding all duplicate files (either by name or file size and type) under Linux/Unix would be pretty quick.

I just wanted to point that out.

0blique

Wouldn't that mean you could boot a Linux distro and then mount the drive and search for duplicates?

By the way, how would you do it?

Doc

0blique wrote: »

Wouldn't that mean you could boot a Linux distro and then mount the drive and search for duplicates?

By the way, how would you do it?

something like this (probably wrong, don't have access to my linux box right now):

cd <root dir you want to start searching from>
find /. <regexp for the file type> -printf %f %k %p | sort +0 -1 | awk <some awk script here, don't really care to write it>

Honestly instead of the awk script I'd probably just put it out to a file and find duplicates with python or something. The point is that you could dump the results of a search to a file, then run a script to find dupes instead of having to look through thousands of files by hand.

Edit: I'd forgotten about the find command in cmd that still exists. Let me whip something up...

Doc

dir -n <regex for your file type> /s > files.txt

then you have to clean the output because it puts in a bunch of extra garbage, and do the same thing as you did with linux, write a small script to find identical fields on consecutive lines.

bluekirby

This would really be useful for freeing up space when I have backups of files stored on an external hard drive.
---

Yummah

Veevee

Doc wrote: »

dir -n <regex for your file type> /s > files.txt

then you have to clean the output because it puts in a bunch of extra garbage, and do the same thing as you did with linux, write a small script to find identical fields on consecutive lines.

Can you write this script for us too?

Also, I wouldn't mind some cake either.

ArcSyn

I am interested in this as well. I'm too lazy to export things to txt and write scripts though..

I know Norton back in the 80s and 90s had a compare and contrast and find duplicates built into their programs, but I don't know about now.

If anyone finds a good one, report back and let us know, because I'm surprised there aren't a few out there already for sorting pictures and music and just plain files in general with how large hard drives have gotten lately.