Find duplicate mp3 and flac files but I only want to delete the mp3s.

[kinglifer]

I been lurking for years on here. Got help and to be honest never said much.
My collection is become too much and I need some space and I been searching on a fix to cure my problems.

first I have tons of duplicates. in my audio and video folders.
Audio
I would like to simply find all duplicates (if I have 2 or more mp3's keep the highest quality.) then...
If I have mp3 or Flac keep the highest quality. (flac)

Video (music videos btw)
If the titles are the same keep Mkv or (offical video)

any help I would appreciate.

[void]

Please try Find duplicates with Everything 1.5.



Perform your audio search, for example:

ext:mp3;flac

For a quick guide to possibly duplicated files, include the following in your search:

ext:mp3;flac dupe:size



To compare content, include the following in your search:

ext:mp3;flac dupe:size;sha256

This will take a very long time to gather sha256 information.
(the sha256 information is only gathered for files with the same size)

[kinglifer]

Thank you for this. I should have been a bit more clear.
No way would the files be the same. Flac and Mp3. That is for sure. almost 100% they would not be.

I have a similar issue with Mp4 and MKV files (videos)

The reason this happened is due to software I use and did not realize it was pulling both types at the same time.

[void]

Searching for length duplicates in Everything 1.5 might help.

ext:mp3;flac dupe:length
ext:mp4;mkv dupe:length

To find dupes with second resolution (instead of 100nanoseconds):
length_dupe_tolerance
-or-
ext:mp3;flac add-column:column-a column-a:=format-duration($length:,"s") dupe:column-a



Would the filenames be the same?

For example:
song1.mp3
song1.flac

Find mp3 files where a flac exists with the same stem:
regex:^(.*)\.mp3$ fileexists:\1.flac

[kinglifer]

File would be something like this

Fine Young Cannibals - The Raw & The Cooked - 01 - She Drives Me Crazy.flac

01 - She Drives Me Crazy.mp3
==============================
All would be in folders in order

Fine Young Cannibals
|
__ The Raw & The Cooked

So with this information I'm hoping I can see which has duplicates .

[void]

regex:^(.*)\.mp3$ sibling:$1:.flac

To list the flac files too:
<regex:^(\d+" - ".*)\.mp3$ sibling:$1:.flac> | <regex:(\d+" - ".*)\.flac$ fileexists:$1:.mp3>
(sort by name for the best results)

[kinglifer]

Please don't get upset
But what does the first does specifically?

[void]

The following will search for mp3 files and capture the filename:
regex:^(.*)\.mp3$

This will match:
01 - She Drives Me Crazy.mp3

and capture the text: 01 - She Drives Me Crazy



The following will search for flac files with the captured text in the same location:
sibling:$1:.flac

$1: is replaced with the captured text:
sibling:"01 - She Drives Me Crazy.flac"

sibling:

[kinglifer]

thank you so much for the reference link!
That is very nice of you. I had no idea what to "search" for help.

[kinglifer]

The following will search for mp3 files and capture the filename:
regex:^(.*)\.mp3$

This will match:
01 - She Drives Me Crazy.mp3

and capture the text: 01 - She Drives Me Crazy



The following will search for flac files with the captured text in the same location:
sibling:$1:.flac

$1: is replaced with the captured text:
sibling:"01 - She Drives Me Crazy.flac"

sibling:

Just fine tuning things. (I was traveling so I couldn't do it before. I am home now). This is working and WOOOH... but it isn't exactly and obviously I am not equipped to know how to fine tune it. So thank you. I am thankful.

Doing this: x:/music videos/ | x:/music regex:^(.*)\.mp3$ groupcolors:
will bring back results:
X:\MUSIC\Our Lady Peace\A Decade (2006)\Our Lady Peace - A Decade - 12 - Somewhere Out There.mp3
X:\MUSIC\Our Lady Peace\A Decade (2006)\Our Lady Peace - A Decade - 13 - Innocent.mp3
^^^^^
As you can see these are not duplicates and this is not a complaint. what I would want is more like this:
X:\MUSIC\Our Lady Peace\A Decade (2006)\Our Lady Peace - A Decade - 12 - Somewhere Out There.mp3
X:\MUSIC\Our Lady Peace\A Decade (2006)\Our Lady Peace - A Decade - 12 - Somewhere Out There.Flac
X:\MUSIC\Our Lady Peace\A Decade (2006)\12 - Somewhere Out There.mp3
^^^^^
To me these are all duplicates (In my situation)

This is also what I am trying to do.
X:\MUSIC VIDEOS\Our Lady Peace\Somewhere Out There.mp4
X:\MUSIC VIDEOS\Our Lady Peace\Somewhere Out There.mkv
X:\MUSIC\Our Lady Peace\A Decade (2006)\Our Lady Peace - A Decade - 12 - Somewhere Out There.mkv
^^^^^
To understand why... software I had, grabbed videos and placed them into album folders (This seems to be a favorite thing for users that use Plex)
However I moved most to a folder called MUSIC VIDEOS with the artists name only. (No need for a folder for album because I use videos for like a MTV type player on my TV. Nothing to do with Plex).
So my goal is to find music videos (in this scenario) in the music folder and if there is duplication in the MUSIC VIDEO folder I will delete the ones in the music folder. If there isn't duplication I will simply move all the music videos to a "MUSIC VIDEO" folder under the name of the artist. Problem solved!

Now I know Everything to my knowledge can not do this automatically. I am just trying to limit the collateral damage. I got over 2tb of folders. I get anxiety just looking at the results lol.

[NotNull]

The serach @void specified ( regex:^(.*)\.mp3$ sibling:$1:.flac ) will list all mp3 files that have a flac with the same name in tthe same folder. Those mp3's can therefor be deleted or moved elsewhere.

Try the following for your mp3/flac files:

Phase 1

• Create a new folder X:\FALLBACK
  Later on, this folder will contain the files that are no longer needed.
• Disable Match Path (in the Search menu)
• Paste the following search in Everything's search bar:

• Code: Select all

  X:\music\   ext:mp3    regex:^(.*)\.mp3$   sibling:"$1:.flac"

  This will list all mp3 files that have a matching flac file in the same folder
• As a sample, check if this is indeed correct for a couple of mp3's (in Explorer).
• If all seems OK, continue
• Select all mp3 files (CTRL + A)
• Menu: Edit => Advanced => Advanced Move To Folder
• Old format: X:\%1
• New format: X:\FALLBACK\%1
• Check New Full Paths
• Press the OK button
  (these Move actions can be undone if needed, so the system can be brought back to its original state)
• Ready with Phase 1

This should clean up a significant portion of your duplicates.
Please report results so we can continue with phase 2

[NotNull]

For a quick overview of how much work still has to be done, paste the following in the search bar:

Code: Select all

X:\MUSIC\   ext:mp3;flac   regex:"(\d{2} - .+?)\.[^.]+$"   addcolumn:regmatch1;audiobitrate   dupe:regmatch1

(assuming all your music files use 2 digits to indicate the position, like Fine Young Cannibals - The Raw & The Cooked - 01 - She Drives Me Crazy.flac

[kinglifer]

For a quick overview of how much work still has to be done, paste the following in the search bar:

Code: Select all

X:\MUSIC\   ext:mp3;flac   regex:"(\d{2} - .+?)\.[^.]+$"   addcolumn:regmatch1;audiobitrate   dupe:regmatch1

(assuming all your music files use 2 digits to indicate the position, like Fine Young Cannibals - The Raw & The Cooked - 01 - She Drives Me Crazy.flac

This is a really neat command... LOVE It. I can do this for two folders as well?
I assume I can change mp3;flac with mp4;mkv as well They do not have numbers in some of them though.

You guys are awesome, seriously.

[kinglifer]

The serach @void specified ( regex:^(.*)\.mp3$ sibling:$1:.flac ) will list all mp3 files that have a flac with the same name in tthe same folder. Those mp3's can therefor be deleted or moved elsewhere.

Try the following for your mp3/flac files:

Phase 1

• Create a new folder X:\FALLBACK
  Later on, this folder will contain the files that are no longer needed.
• Disable Match Path (in the Search menu)
• Paste the following search in Everything's search bar:

• Code: Select all

  X:\music\   ext:mp3    regex:^(.*)\.mp3$   sibling:"$1:.flac"

  This will list all mp3 files that have a matching flac file in the same folder
• As a sample, check if this is indeed correct for a couple of mp3's (in Explorer).
• If all seems OK, continue
• Select all mp3 files (CTRL + A)
• Menu: Edit => Advanced => Advanced Move To Folder
• Old format: X:\%1
• New format: X:\FALLBACK\%1
• Check New Full Paths
• Press the OK button
  (these Move actions can be undone if needed, so the system can be brought back to its original state)
• Ready with Phase 1

This should clean up a significant portion of your duplicates.
Please report results so we can continue with phase 2

Got a File Operation Error: 00000002
It could be me... I did not do [*]Old format: X:\%1 but I did [*]Old format: X:\MUSIC\%1 assuming it was a typo

[void]

Error 2 is path not found.

Please try forcing a rebuild:

• In Everything, from the Tools menu, click Options.
• Click the Indexes tab on the left.
• Click Force Rebuild.
• Click OK.

Please try the move again.

I wonder if there's a really long path name in there.
Does the following search find any results:

X:\music\ ext:mp3 path:len:>=260

I did not do [*]Old format: X:\%1 but I did [*]Old format: X:\MUSIC\%1 assuming it was a typo

Either will work.
Using X:\%1 will create: X:\FALLBACK\MUSIC\...
Using X:\MUSIC\%1 will create: X:\FALLBACK\...

[kinglifer]

I wonder if there's a really long path name in there.
Does the following search find any results:

X:\music\ ext:mp3 path:len:>=260

34 items with a long path returned.

The serach @void specified ( regex:^(.*)\.mp3$ sibling:$1:.flac ) will list all mp3 files that have a flac with the same name in tthe same folder. Those mp3's can therefor be deleted or moved elsewhere.

Try the following for your mp3/flac files:
This should clean up a significant portion of your duplicates.
Please report results so we can continue with phase 2

Returning zero now! I am def ready for phase 2!

[kinglifer]

From the awesome assistance here I put one and one together and did this...

X:\music\ ext:mkv regex:^(.*)\.mkv$ sibling:"$1:.mp3"

So this returns music videos in my music library which should not have been there. Perfect.

Now my music video folder has just artist name as the folder and inside is the videos. The music would be

Robin Thicke-Blurred Lines.mkv

as it is now
it is in a folder

X:\MUSIC\Robin Thicke\Blurred Lines (2013)\

called

Robin Thicke - Blurred Lines - 01 - Blurred Lines.mkv

I know there is another software that does renaming but it would be nice to somehow get them into a temp folder like you set up for music but not so nested like that. just the folder of the artist and then the videos that was found in the music folder.

as it is now I have 483 videos nested quite neatly actually but not how I want them in a music folder.

[NotNull]

I did not do [*]Old format: X:\%1 but I did [*]Old format: X:\MUSIC\%1 assuming it was a typo

It was not a typo. This format guarantees that the move action can be undone:
X:\music\song.mp3 and X:\videos\song.mp3 would otherwise overwrite each other in the Fallback folder.

Returning zero now! I am def ready for phase 2!

That means a phase 2 for audio file is not needed at the moment.
There might still be filenames that don't follow this naming pattern, but the bulk should be ready now.
If there are such files remaining, please report their naming convention.

To see the amount of disk space you will be saving, enter the following query in the Everything serach bar:

Code: Select all

exact:X:\FALLBACK

The amount will be shown in the Size column.



Video

So my goal is to find music videos (in this scenario) in the music folder and if there is duplication in the MUSIC VIDEO folder I will delete the ones in the music folder. If there isn't duplication I will simply move all the music videos to a "MUSIC VIDEO" folder under the name of the artist. Problem solved!

Phase 1:

For all video files in the following format:
X:\MUSIC\Robin Thicke\Blurred Lines (2013)\Robin Thicke - Blurred Lines - 01 - Blurred Lines.mkv

• Paste the following search in the Everything search bar:

  Code: Select all

  "X:\music\*\*\* - * - * - *"     ext:mkv;mp4
  

• Select all files ( CTRL + A)
• Menu: Edit => Advanced => Advanced Move to Folder
• Enable the Regular Expressions checkbox
• Old Format: ^X:\\MUSIC\\([^\\]+)\\[^\\]+\\[^-]+- [^-]+- [^-]+- (.*)$
• New Format: X:\\MUSIC VIDEOS\\\1\\\2
• Check New Full Paths
• If all looks OK, press the OK button (this action cannot be undone easily, so check carefully)
• If this file already exists in X:\Music Videos, select Skip when asked what to do with this file
  (I think pressing SHIFT when selecting Skip will skip all similar, but not sure)
• Back in Everything's main window, the list will no longer show the moved files. Just the skipped ones. They are duplicates and can be moved to the FALLBACK folder:
• Select all files ( CTRL + A)
• Menu: Edit => Advanced => Advanced Move to Folder
• Old format: X:\%1
• New format: X:\FALLBACK\%1
• Check New Full Paths
• Press the OK button

You mentioned that some voideos don't have a number in their name.
What format do they use? Robin Thicke - Blurred Lines - Blurred Lines.mkv?
Are they few and far apart or is it worth to do something similar as above?


(Phase 2 will weed out the .mp4's if a .mkv is available)

[kinglifer]

Sorry had a serious personal matter happen. (Sigh)

ok

You mentioned that some voideos don't have a number in their name.
What format do they use? Robin Thicke - Blurred Lines - Blurred Lines.mkv?
Are they few and far apart or is it worth to do something similar as above?

The ones that do not have a number look like this Robin Thicke - Blurred Lines.mkv or Robin Thicke - Blurred Lines.mp4
that is the only two variants.

so I can have duplicates like this Robin Thicke - Blurred Lines - Blurred Lines.mkv / Robin Thicke - Blurred Lines.mkv or sometimes Blurred Lines.mkv in a Robin Thicke folder.

Thank you. Like you saved me hours so far just in the audio part.

[NotNull]

Please let us know how phase1 for videos went.

[kinglifer]

I am still here I will send my results when I get back home later tonight. I do apologize, unexpected situation happened.
I just remote desktop into my pc and this worked perfectly. INSANE.
I honestly am curious how do I learn these skills you have. It is honestly awesome for data management. Thank you.
Honestly Thank you.


Soo I did everything to get MKV files out of the artist folder into the music folder. I did see duplicates 80 or so and a lot moved. 400 or so.

I would like to continue. (bookmarked!). A question I have some that ended up like this...
X:\MUSIC VIDEOS\Alanis Morissette\Alanis Morissette - You Oughta Know.mkv
and a few like this...
X:\MUSIC VIDEOS\Alanis Morissette\You Oughta Know.mkv (just for example...)

can Everything check folders and if the folder name is not in place add "Alanis Morissette - " without quotes (for this example) and if it is already there skip?

[kinglifer]

Please let us know how phase1 for videos went.

It went well Just the fall back put all videos in the folder. I was hoping it would nest them in the same folder as the original was in.
Prince-|
Purple Rain-|
Prince - I Would Die For You.MKV

[NotNull]

The fallback folder is just that: a fallback. It contains the duplicates of the (correctly named) files in X:\MUSIC VIDEOS\.
If all went well, the files in X:\fallback can be deleted.

Do you have a specific reason why you want to rename the files in X:\fallback too?

I honestly am curious how do I learn these skills you have.

It is mostly about finding patterns that describe the files. In this case " - " and "\" are good 'delimiter' of the various 'components' that describe your files. The trick is to find a pattern that matches your files, but don't match unwanted files.

About 80% of the techniques to translate this into "rename code" I learnt on this forum.
Just by trying to answer questions like yours and learning from other people's answers.


Tomorrow (no time today) renaming/moving files like Robin Thicke - Blurred Lines - Blurred Lines.mkv

[kinglifer]

I think maybe I am setting it up wrong?
I assumed the fallback was files I could move.
These are duplicates? Just getting slightly confused.
No problem rest enjoy your evening.

[NotNull]

Video files (mkv mp4) were moved out of your X:\music\ folder.
The main ones are now in X:\music videos\. They were also renamed into the naming structure you prefer.
The remaining video files from X:\music\ are duplicates and are moved to x:\fallback.